Ask HN: Google Chrome heuristic warnings pose threat to our business
Many businesses use our Enterprise File Sharing Product called FileCloud (http://www.tonido.com/filecloud). Think of it as a self-hosted alternative to Dropbox. With the latest Chrome update, the browser is showing phishing warning (http://patch.codelathe.com/foruser/phish_1.jpg) with our installations. The warning is not based on the domain and it appears in our different customer installations. It has to be heuristic based because it generates warning even on a debug/local webpage. The chrome browser heuristically decides our login page as a phishing page and gives the wrong warning. We are trying to find if there are any published "guidelines" as to legitimate web pages should NOT be doing to trigger these? Either there should be clear methods to resolve these warnings or Chrome should avoid doing this blanket-so-called-protection racket.
Because of Google’s missteps, our reputation as well as customer reputation got a hit. We have spent countless hours in our resources to see what is going on and all thing points to heuristic decision making by Chrome browser. There is no way to contact Google Chrome team to resolve this issue. We have lost few large deals. Now all our support team is pretty much focused on this issue and fielding queries from our customers.
Since our UI code (Developed in GWT) is common between our Enterprise and Consumer product (Tonido), if we this error start appearing in our consumer version (half a million users) it is an EXISTENTIAL RISK to our company that we have built over 5 years.
We have 2 questions.
1. How to get in touch with Chrome team and solve the issue?
2. Are there any legal avenues or precedence to force Google to take action and claim compensation for lost business?
Please provide us with your suggestions.
P.S: It is happening to our software today. It may happen to your products tomorrow.
84 comments
[ 3.6 ms ] story [ 154 ms ] threadOther contact forms: Mailing Lists: http://www.chromium.org/developers/discussion-groups IRC Channel: http://dev.chromium.org/developers/irc
before downvoting, take the time to explain how this would be different from old good Google suing Microsoft just for not making their product use Google easier than it already allowed.
and i'm getting downvoted alright, and without any attempt at fulling my request, no less
The next step for us is to hire PR and go public. We are spending 1500$ per month on google adwords now.
May be use that money to get some legal help. In a physical world it is a clear public defamation case.
Google's failure to respond to issues like this is appalling, and it's probably going to take a lawsuit and public embarrassment to get them to stop being evil.
The irony is we use GWT for our UI which is used by many google products. when we search in web the closest issue we found if from a joomla forum: http://forum.joomla.org/viewtopic.php?f=621&t=802284
Use Google Webmaster Tools for your product site and check for issues: https://www.google.com/webmasters/tools/home?hl=en
Try to come up with a reason why this may not be a false positive. Perhaps you have trademark issues? etc.
More info:
http://blog.chromium.org/2008/11/understanding-phishing-and-... This includes the URL of the website you are visiting, as well as the URL of any included resources (such as included JavaScript or Adobe Flash movies)
https://support.google.com/chrome/answer/99020
https://www.usenix.org/legacy/event/hotbots07/tech/full_pape... [pdf] The Ghost In The Browser. Analysis of Web-based Malware (a paper to make this post interesting to others)
Shows up now with Chrome v27.0.1453.116
No issues.
Our software is little different. It is a self-hosted software. It is hosted by our customers under different domain names in their infrastructure. So it is not the same domain or URL.
For Example:
Customer 1: fileshare.abcplumbing.com
Customer 2: dataanywhere.peterlawfirm.com
Thats the real problem here. It affects our customer installations under different domains. To some extent, we are fine if google is blocking one domain because somebody in the domain is sharing malware. The issue here is different.
The warning appears even in local IP/debug page.
Another thing it could be is if you fetch any assets at all from your domain, and the domain is blacklisted, that causes the warning regardless of the page where it is hosted.
This is, of course, just a random guess to throw into your brainstorm.
maybe you caught a malware on your computer. did you try from different machines?
It is like playing proverbial whack-a-mole. Every update of chrome can potentially change their "heuristic" that thinks it has "found" a phishing attack.. and we have to scramble to see what the heck caused it and fix it.
This would be funny if it wasn't so detrimental to a business. I will get the dev to recreate this on dev1 and post it.
[5760:1799:0701/150256:VERBOSE2:phishing_classifier_delegate.cc(211)] Phishy verdict = 1 score = 0.548927
[5751:1799:0701/150256:VERBOSE2:client_side_detection_host.cc(447)] Feature extraction done (success:1) for URL: http://dev1.codelathe.com/ui/core/index.html. Start sending client phishing request.
[5751:1799:0701/150256:VERBOSE2:client_side_detection_host.cc(415)] Received server phishing verdict for URL:http://dev1.codelathe.com/ui/core/index.html is_phishing:0
You may need to restart the browser between edits, as it seems to cache the classifier results by URL. It also skips classification for hosts with private IPs, I had to jump through some hoops to test.
Must be nice to dream up some "algorithm" and push it out.. sigh
[5570:1799:0701/133949:VERBOSE1:client_side_detection_host.cc(221)] Instruct renderer to start phishing detection for URL: http://dev1.codelathe.com/ui/core/index.html [5579:1799:0701/133949:VERBOSE2:phishing_classifier_delegate.cc(238)] Not starting classification, no Scorer created. [5579:1799:0701/133950:VERBOSE2:phishing_classifier_delegate.cc(238)] Not starting classification, no Scorer created. [5570:1799:0701/133954:VERBOSE2:client_side_detection_service.cc(255)] Sending phishing model to RenderProcessHost @0x7aa18a00 [5570:1799:0701/133954:VERBOSE2:client_side_detection_service.cc(255)] Sending phishing model to RenderProcessHost @0x8043d620 [5579:1799:0701/133954:VERBOSE2:phishing_classifier_delegate.cc(283)] Starting classification for http://dev1.codelathe.com/ui/core/index.html [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlTld=com = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageImgOtherDomainFreq = 0 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlOtherHostToken=dev1 = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlPathToken=html = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageLinkDomain=tonido.com = 1 [5574:1799:0701/133954:VERBOSE2:phishing_classifier_delegate.cc(275)] Not starting classification, last url from browser is , last finished load is chrome-extension://jpjpnpmbddbjkfaccnmhnkdgjideieim/background.html [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlPathToken=core = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageTerm=password = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageHasTextInputs = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageExternalLinksFreq = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageHasPswdInputs = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageSecureLinksFreq = 0 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageTerm=connexion = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlDomain=codelathe = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlPathToken=index = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageTerm=account = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageHasForms = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageNumScriptTags>1 = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageNumScriptTags>6 = 1 [5579:1799:0701/133954:VERBOSE2:phishing_classifier_delegate.cc(211)] Phishy verdict = 1 score = 0.548927 [5570:1799:0701/133954:VERBOSE2:client_side_detection_host.cc(447)] Feature extraction done (success:1) for URL: http://dev1.codelathe.com/ui/core/index.html. Start sending client phishing request. [5570:1799:0701/133954:VERBOSE2:client_side_detection_host.cc(415)] Received server phishing verdict for URL:http://dev1.codelathe.com/ui/core/index.html is_phishing:1 [5570:1...
mixing different languages might be bad. try changing all the page text (and corresponding content-language header) to the localized language instead of just changing the submit button. maybe this have the classifier use the contextual meaning of "connexion".
First of all we see that this so called phishing detection filter's code is found at http://src.chromium.org/svn/trunk/src/chrome/renderer/safe_b...
Second, this code and the logic it employs is really bull.
The world wide web is not a kiddie playground especially for a browser, and especially for a plugin whose's job is to detect phishing. The way Chrome's anti-phishing works is to use several foolish measures that mean nothing in the real world and then 'punish' and push websites into oblivion when someone crosses these arbitrary sets of rules.
The way the plugin appears to work is to look at various things * The type of URL (IP vs domainname, number of subdomains, size of the subdomain names, the strings in the Path URL) * Whether the page contains form data * Whether the page contains password input box * Whether the page contains checkboxes/radio boxes * Whether the page text contains some terms (in this case 'connexion') * Whether page has links/images to other domains
and so on.
None of these are ANY indication of phishing behavior and if this set of quackery based logic is what we see from Google Chrome, where else can we go to really feel safe and protected?
I'd take bets that those criteria show a correlation to phishy sites. Especially if you combine those metrics together.
Is it perfect? No. Does it produce false positives? Yes. Is it beneficial on average? I think so.
PS: Since you have found the relevant file in the open source project (or 'kiddie playground' - as you like to call it), why don't you supply a superior implementation with less "foolish" measures?
Browser is the window through people sees the world. That’s the reality we live in. In our target market, Google chrome holds 40% market share. Because of its stupid categorization, in one stroke Google harmed our reputation and the reputation of companies we serve. It is not a simple browser compatibility issue. Google chrome is telling the world our software is phishing software while we are not. What is the recourse here?
We don’t care what Chrome’s algorithms are. But the results are not factual and it harms our business. "One cannot escape saying hey that is our algorithm. We don’t do evil…" Remember.
But I don't think that I am trivializing things. The fact is, that phishing sites are causing a real pain (as in millions of dollars lost by the victims, hundreds of thousands of computers becoming zombies, etc). All major browsers are trying to mitigate these risks by implementing phishing & malware filters. None of these implementations are perfect (you probably know a bit or two about bugs in software development).
But on average these filters have a positive ROI - especially for the target market (which is Joe WebUser and sadly NOT your company - or mine for that matter). The costs of a false positive ("I'll go & find that information on another site") far outweigh the costs of a false negative ("I put my login+password into this legitimate looking website and now I can no longer access PayPal").
My point is that if you are going to design a system to identify bad websites it better be fail safe otherwise it is going to cause a lot of hurt.
The message shown in the browser for a phishing warning is the same as when a website has an invalid SSL certificate. The first is vaguely accurate, the latter is 100% accurate and no one is going to argue if the warning is needed. Both show the mind chilling warning no sane user will click through.
I am more interested in removing the phishing filter than in writing a phishing filter.
Anyways, with a 'closed' server component also in the mix, what option is there to provide any implementation.
IMHO, I think that doing things for the 'benefit of most' will lead to eroded freedoms for all over time.
PS: 'Supply a better implementation' is not an answer to writing poor code and hoisting on the world.
Yes. Lets apply this everywhere. Lets electrocute folks based on "heuristics" because there are no other way to find out "bad guys".
It is nice to act as an arbiter and spout philosophy isn't it?
If you really do think that there is no other better way then I guess there is no more point arguing about this.
The problem is 1. No clarity on what constitutes a problem. 2. No way to officially contact to clear up a problem
resulting in possible irreparable loss of business.
So, if you insist on interesting and orthogonal "analogies".. please carry on.
> So, if you insist on interesting and orthogonal "analogies".. please carry on. If it was not clear, I was trying to describe a possible issue with you "lets apply this everywhere" argument.
The two arguments you just put forward, are nowhere close to what you said in the comment I replied to. Yes, there are issues with the current implementation of it, which is very similar to how spam detection/prevention systems work at the moment. Yes, there can be improvements to it. There can be improvements to everything. Yes there is high chance of false negatives in the current system, but this is a problem where false positives can be just as disastrous. If we cannot agree with that, then do not think it is worth continuing this discussion.
Now if you check the top comment on the thread, I believe the communication channels have already been set. They did not work for you as promptly as you would want them to, that's a different issue. But there definitely exists an official contact to clear up the problem - your colleague seems to be aware of it. The lack of clarity of the reasons has been marked as intentional and has been discussed elsewhere on the thread.
It was poor of me to use snark instead of clearly stating my stance, but the stupidity of analogy that you are blaming me for, is not much different from what I was trying to mock.
Every update to the browser can potentially change the model that affects a large number of the users and the only way to figure out the problem is using some sort of trial and error method.
This would have been fine if the product in question is a niche product or a exotic browser. But the fact of the matter is, with Chrome (being one of the dominant browser) and Google being the product owner, the reach of Google's opinion is far reaching and can easily destroy a product (akin to killing a person based on some assumption).
Also note that, the "communication channels" listed earlier were completely useless for this type of problem where the client side is throwing the error (Not related to a specific domain or even url included in the page).
Understand that, being a commercial product, ALL possible methods were tried (obviously) to resolve it by using those methods and could not resolve it. You can see that, this specific instance gets triggered by simply having a button with the name "Connexion" instead of "Login" (purely detected by backtracking the changes).
So the frustration is not meant to belittle Google's effort at combating spam/fraud but to point out the effect of such wide ranging blanket solutions.
While "Collateral damage" is a very nice way to de-sanitize and make things palatable for all parties involved except those getting to be the "Collateral damage".
At the end of the day, I am sure folks understand that Google being Google can do what they want and probably even bury the whole issue from getting any traction.
Wasn't your site explicitly whitelisted?
Also, dev1.codelathe.com was re-setup specifically to trigger the warning (It was determined that if the login button has the keyword "Connexion", it was pushing the phishing score past 0.5)
The main thing is, if there is a clear way to contact the team responsible for this to resolve such issues, that will be the best way for anyone with similar problem and at this point there is no such avenue.
That, precisely is the issue. We can only speculate as to what might be good or bad. There is no way to really know is correct (and infact why is it even the business of a browser to determine that). In a lot of situation, it is not something the product gets to decide. For example, this got triggered when a customer added translations to the application (which changes that button).
.
in your case, non-translated "Account" and "Password" texts in a french corpus are most probably much much more common than a wrongly-translated french "Connexion" in an english corpus...
that said, i do not know if chrome is really considering the language or not, but i certainly would hope so. :)
Since it looks like it's a model trained offline on known phishing sites, unfortunately I think your best bet is tweaking until you fall under the threshold and (if you're feeling magnanimous) filing a bug on Chrome with an example of how the current model is flawed (though if the page is working in dev channel, something may already have been fixed).
That sucks, sorry :(
[1] https://code.google.com/p/chromium/codesearch#chromium/src/c...
Can you remove the "core" part of the url? => UrlPathToken=core = 1
You could try another file extensions for the page (don't now if this is possible with gwt). => UrlPathToken=html = 1
The "powered by" link seems also problematic => PageLinkDomain=tonido.com = 1
Chrome user$ grep phish chrome_debug.log | grep -e "UrlPath" -e "PageTerm"
[5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlPathToken=html = 1
[5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlPathToken=core = 1
[5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageTerm=password = 1
[5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageTerm=connexion = 1
[5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: UrlPathToken=index = 1
[5579:1799:0701/133954:VERBOSE2:phishing_classifier.cc(192)] Feature: PageTerm=account = 1
It is essentially engineering how things should be developed, which still could be tolerable if there are guidelines.
If an average user sees a red page indicating risk to a page, then that site/page is essentially killed.
Trace showing server overriding the "Phishyness" verdict of the client
[5760:1799:0701/150256:VERBOSE2:phishing_classifier_delegate.cc(211)] Phishy verdict = 1 score = 0.548927
[5751:1799:0701/150256:VERBOSE2:client_side_detection_host.cc(447)] Feature extraction done (success:1) for URL: http://dev1.codelathe.com/ui/core/index.html. Start sending client phishing request.
[5751:1799:0701/150256:VERBOSE2:client_side_detection_host.cc(415)] Received server phishing verdict for URL:http://dev1.codelathe.com/ui/core/index.html is_phishing:0
I'm sure they will contact you via regular customer support channels (ie. not HN) if they need any more info to debug the problem.