I have gained admin access to numerous GCloud Organizations by accident
in Google Cloud, you can assign admin, billing, etc to a google group.
Years ago I made a google group for google cloud administration
A company in Spain, a bunch of startups, etc have added that google group (by accident) as an IAM user with varying level of roles attached
I now have billing access to one account, admin access to another, can just hop into the database of at least two of the accounts
I try to reach out to google support but because I don’t have “business” or “enterprise” level support I can’t even submit a ticket
I’m trying to let them know but can’t, they do t do chat, no phone number, even billing contact is an automated chatbot only
GCloud should have like “emergency reach out to a person” link or something
131 comments
[ 389 ms ] story [ 4062 ms ] threadDidn’t poke around, was more like what the heck is this?
I only noticed because I logged in and the page defaulted to Spanish (it picked the first org, which happens to be a Spanish car company)
Then I noticed in the drop downs. I actually thought I was hacked, then realized what was going on.
Still trying to find a way to get ahold of Google lol
[1] - https://www.businessinsider.com/guides/tech/how-to-contact-g...
[2] - https://googleprojectzero.blogspot.com/
[3] - https://about.google/contact-google/
Good luck. You're trying to do the right thing but if they lawyer you, remind them they added you not you added them.
Don't go into these accounts at all. Not even to try and help/contact them. Laws about this are very vague and no one within the ORG would want to admit that they made a mistake by adding you.
Amazon’s support has gone above and beyond for me over the years in ways I didn’t even expect or ask them to.
In comparison, I agree with you that Google’s support is useless.
My experiences with AWS support have actually left me with a positive impression of the platform, while my experiences with Google reinforced that they don’t know how to do support. At all.
Want to guess where our 8-digit cloud spend goes?
Outsourcing inevitably creates a firewall between engineering and support that shouldn't exist though.
In a properly functioning org, support has a way to escalate quickly to engineering if it's confirmed "This is broken." Engineering in turn uses those incoming requests to recognize flaws in their own products.
Outsourcing creates "Hide behind the SLAs and remain ignorant of any issues you've created" barriers that will ultimately sink a company.
But to me, the question has always been "Do I think I'm omniscient as an engineer? Do I think I can imagine every way the customer is going to try and use this product? Every way it can interact with other systems? Every quirk of a specific customer environment/dataset/etc.?"
Well, if not, then good news! The support org should be capturing, categorizing, documenting, and forwarding all those cases to me.
And each case is an opportunity to make the product better!
From their subprocessors list it looks like it was most likely "WEBHELP ROMANIA SRL"
What's weird is that GCP charge the same but apparently don't deliver
I submitted a ticket to the support team advising them in painstaking detail the steps needed to reproduce this vulnerability. They could also look at my account and see that I got stuff without paying.
A couple days later I got a reply from a support manager that my concern wasn’t valid and there was no bug.
The next week I happened to be at a conference where the company in question was a sponsor. So, I visited their booth and spoke with the VP of Eng. He asked me to forward the ticket to security@. Within 8 hours I got a reply from them saying that they had fixed the bug.
I guess I’m saying that even if Google let you submit a support ticket it might get ignored because they aren’t trained to deal with security reports.
https://bughunters.google.com/
Also, it doesn't shock me that somebody got a common group name early on in an internet-scale service's lifecycle. I've had a couple such experiences. Simple example: in the early days of Google Hangouts, you could choose your own meeting name in the URL. I chose "compass" for a meeting and accidentally landed in a meeting of Google engineers who were very surprised by my appearance. Fortunately my meeting was a meeting I had arranged so I beat feet and changed my URL to the default auto-generated URL before the rest of my participants arrived.
Or, actually have support, but that's not Google's style.
https://securitytxt.org
For example: https://www.google.com/.well-known/security.txt
I understand some scrappy startups like Google don't have the resources to have someone review security incident reports that come through a web form, but maybe they should if they want to be a legit cloud provider?
Googling "report google cloud security issue" does not turn up productive results. Compare to what you get when you google "report aws security issue."
That seems to suggest that Microsoft takes all security reports seriously even if most turn out to be bogus.
[1] https://devblogs.microsoft.com/oldnewthing/20221004-00/?p=10...
[2] https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31...
From misc. articles I've seen (mainly posted here on HN; I don't buy MS products) MS dismisses bug reports as unimportant and sometimes takes an extremely long time to address known security vulnerabilities.
This VM escape was initially reported as an RDP bug that MS dismissed as unimportant, until it was used as a VM escape against their hypervisor.
https://www.bleepingcomputer.com/news/security/microsoft-ign...
The (in)famous pass-the-hash bug in windows is an example of MS not addressing serious security issues in a timely manner. Windows treats a password hash as equivalent to the password, so you don't even need to crack hashed passwords you've collected from e.g., the registry to authenticate to windows services (MS "protected" against this attack purely client-side). Microsoft acknowledged the issue was real more than a decade before even attempting to fix it.
Apparently it was a difficult bug that included design failures, but over 10 years and multiple versions of windows for an exploit this severe?
A couple days ago a Google Cloud container escape made HN front page. Comments on that article indicated Microsoft Azure had recently suffered the same, but while Google only allowed access to other containers owned by the same tenant, Microsoft's escape allowed access to all tenants on the same host. Google added a second layer of safety in case the first failed (a dedicated VM per host per customer to run each costumer's containers). Microsoft YOLO'd. I don't care enough to research these claims beyond noting that at the time I read them, no one had disputed them.
I don't know if Microsoft is overall still worse than its competitors WRT to security (I suspect it is true). But, Microsoft is certainly not an exemplar for how security should be done.
More on-topic with main thread, nonexistent support is kinda what Google is known for?
At least Google now uses abuse@gmail.com for reporting abuse from their infrastructure instead of forcing the reporting party to go through a god-awful web form (when I handled mail at past orgs, I didn't even bother reporting gmail abuse due to the hoops they made you jump through back then; I also used the RFC-Ignorant RBL to punish them and other sites that did not use the RFC mandated email addresses for reporting abuse with a higher bias toward triggering a SPAM tag on their mail).
Perhaps time for an RFC that mandates security contacts?
I can't find any articles for this whatsoever on Google. No matter how many times I include "windows" or "microsoft" (quoted or otherwise) I only get clickbait SEO-spam articles talking about pass-the-hash vulnerabilities in general, not any description or reference to a specific incident in Windows.
Could you please link some article about this so I can read about it?
But thank you for the PDF, it seems like an interesting read!
(non-defanged link for any future visitors: https://www.coresecurity.com/sites/default/files/private-fil...)
That’s an interesting take.
a) Google doesn't care about giving user support for their products even if you pay
b) Over a not-so-long time the survival rate of every Google product seems to drop to zero unless it is related to search and ads.
So, the joke is that the problem would solve itself when google predictably kills this product.
The system is broken if this has happened _multiple_ times to this guy.
Not making the group do confirmation, or even acknowledging the addition is super stupid
I think you have better chances contacting people in the org who added your group to those roles.
1. They should contact the firms involved, make them aware of the situation and then the firms will take a decision on whether to remove or not.
2. They should then look over GCP design and see if there's something that they can do to prevent a reoccurrence of this type of error/mistake
Among other things, I received:
Relevant point being -- every single one of these counterparties had no idea what to do with me responding "I am not the person who you've been talking with about this. They appear to be using my email. Please ask them to update their email."It made me realize how shitty most people are at dealing with anything other than business-as-usual.
If something goes wrong, someone accesses or modifies something that they shouldn't have, you having access is going to be at _best_ confusing to everyone. At worst the cops or lawyers will come calling. Sure you'll _probably_ be able to talk them down, but does that sound like fun?
Or what if someone breaks in to _your_ account and accesses that way? Untangling that mess will not be a good time.
People in the Netherlands and England where I also lived for a short while, are pretty chill with these kinds of things. I can't imagine them doing anything other than thanking you profusely.
I mention this because I'd rather this kind of attitude wasn't imported to Europe.
While I'm not familiar with the nuances of each European nation's computer fraud laws regarding this, I can't imagine this would be any different there. Especially as Cybersecurity becomes an increasingly international concern.
Imagine that, at 5pm on Friday, you discover your IT system has been the target of a huge hack, possibly by russians or north koreans, that they got access to everything, it's been going on for months, and it's certainly a notifiable breach under GDPR.
Would you be chill?
I can say from experience, many people call the cops and lawyers first, and only find the support ticket that first-level support fobbed off with a canned response much later.
Lawyers aren’t chill about anything, and it could be financially ruinous to try find out.
1. One from the party wanting to add the group to their account. Based on a prior comment, sounds like you are prompted to confirm an external group being added as admin.
2. One from the party administering/owning an external google group being requested to be added. Is there any confirmation here?
Without the 2nd confirm, I start imagining security exposures in the family of Ransomware - let's call it "RansomAdd". You randomly add external google groups until you get someone to poke around "too much" and then threaten them with legal action unless they pay up. Ugh.
(And, in the other direction, there should be a request/response flow when you're added to some random project/org you have no interest in, which can make you vulnerable both to legal attacks by the org mistakenly adding you and to phishing.)
In fact, lots of distros now warn when a user attempts certain sudo actions, for similar reasons—mistakes were being made, and adding a little or the right kind of friction could prevent them.
for g in * do wget --recursive "g"; done;
You can also assume that by virtue of you having posted this here and being on the frontpage, it's probably made it to the internal Google SRE IRC chat by now and someone is trying to find a contact. This almost always works :)
Maybe edit your OP with a way to contact you, so that someone can reach out.
In that case no point in following up at all right? Just post on HN and hope someone in the right spot sees it?
> This almost always works :)
That’s the type of SLA one can rely on!
> Maybe edit your OP with a way to contact you, so that someone can reach out.
Having to break online anonymity so that a company can impose the Hollywood rule, “don’t call us, we’ll call you!”, is a truly lousy support structure.
That's how a lot of Google tech support happens. If you get banned by mistake, you have far better luck making noise here or on Twitter vs actually going through support.
We had an app mistakenly banned that we only got human eyes on by calling in favors from old friends who work at Google. It's asinine.
But it's not even the first time this issue was posted here. I'm not sure that approach works with Google.
https://news.ycombinator.com/item?id=34193047
Of course, it would be better if there was an actually supported channel for sending this kind of information, but that's really not the fault of the people that end up finding this stuff and posting it internally (who are often not even related to the problem, posting more of a "hey, anyone know anyone who can help this guy?" message).
FWIW, the security disclosure form I posted will end up reaching a human, which is why I suggested doing that anyways.
I did file a bug bounty hopefully that goes somewhere
I'd be jealous, but then I realized it prob has good uptime whereas using Slack is like a free day off every month with its SLA.
(The box was also useful for a lot of other things, like an Openarena server. We tend to play StarCraft 2 these days though.)
It's the third-party's security team (if there is one, otherwise engineering, contractor hirer, whoever) that should care isn't it?
Eventually i got super fristrated and made a fresh azure trial account for them and boom everything works.
I cannot understand how gcp is so bad at ux and support. Most of the engineers i know at google are the absolute smartest people i know, how in the heck can it be the product experience at gcp is so lousy.
I didn’t even know this hit front page till you said something
I’m just gonna leave the other orgs alone and not doing anything in there until I can figure out a strategy to delete this google group (which I am actually using to manage my own accounts) my accounts are just hobby accounts more than anything, it’s crazy I logged in and found these full-blown business accounts lol
Just insane to me that I don’t have to confirm on my end that I should be the admin, or billing role lol, they can just one way add you…
I think they meant to add their service account and instead added my google group, the URLs are kind of similar
Person abandons old account attached to a group/project, account then hacked, et voila!
It's also probably in breach of GDPR regs that say you should be able to update your own information if it's incorrect.
I will painstakingly change that to not use groups and then delete the group if it lets me
It’s just kinda stupid people are allowed to just add my group with my group not even confirming
I’m looking through these comments to see how I can reach out to google cloud from these links
I do not even want this access…!!!