I'm getting a lot of requests on our servers for "/e8f6b078-0f35-11de-85c5-efc5ef23aa1f/aupm/notify.do" so I can confirm this is in production. I can also confirm they suck at JS.
How about creating /e8f6b078-0f35-11de-85c5-efc5ef23aa1f/aupm/notify.do on your server to notify Comcast users about what their internet service provider is doing? If people started doing that en masse it could bring attention to the problem and with enough publicity get Comcast to reconsider JS injection.
I don't really understand the point of this, either. Couldn't they starting redirecting users to a static page somewhere if there were a real need for a "critical and time sensitive" alert? If the supposed alerts aren't critical enough to justify doing that, use email, IM, RSS or twitter, or even build a custom notification notification app.
Cable companies, IIRC, aren't common carriers the same way a phone company is. They aren't regulated by the FCC the same way phone companies/tv broadcasters are.
Maybe the power company should blink account warnings in morse code through your lightbulbs. After all, it may not be the person paying the bill that is using all of the power....
1. They could email you.
2. They could send you a SMS.
3. They could let you view your bandwidth usage by logging into their site.
4. They could provide an application (desktop or mobile) to keep track of your bandwidth and alert you at certain points.
I use T-Mobile as my mobile carrier and as far as I know they do numbers 2, 3, and 4 that you listed here. I know this because I have received an SMS when I neared my 2GB of unlimited 4G data transfer. I also have logged into their site and used the app on my phone (HTC One S) to monitor my data usage. The phone app even tells you how much data was used by each app and when. It is fantastic. Could that be so hard for Comcast?
My provider (T-Mobile in the UK, using a mobile 3g dongle) send me an SMS, and the connection software has lots of graphs and numbers.
They still send interstitial content warning me that I've exceeded my fair-use limit. It's a bit annoying because I very carefully checked what the limits were before I signed up.
What's worse is that they use weird, broken, IP addresses and horrible proxies for image mangling.
My ISP gives emails at 50%, 80% and of course 100%. They also do options 3 and 4 (no idea about 2) but the emails are so very easy, and knowing you've hit 50% gives you time to mitigate before you get capped.
Your profile reveals you're a former Comcast employee. That's a disclaimer worth posting here.
But yeah, if you have service with Comcast they have your home phone, email addresses, and physical address. They can get in touch with you every way that every company that CAN'T read all of your internet traffic already gets in touch with you.
The method they've chosen is terrible for at least the following reasons:
- The alert will not work on many platforms & devices.
- The alert may not reach the account owner.
- The alert will not work on SSL traffic.
- There is no record that the customer saw the alert (contrast with phone call)
- There are serious privacy issues involved in parsing user's web traffic.
It's extremely bad. The fact that ISP monopolies are not regulated in favor of consumers is slowly going to destroy the openness that has made the web so successful. Instead of giving us raw pipes these monopolies are injecting themselves as proxies where they can monitor, cap bandwidths, shape traffic, censor content, insert messages and even add ads, which Comcast already does when a DNS request is not resolved in a HTTP session.
If you look at what Comcast does on the TV side, things like adding ads to the guide so it's barely usable, you can see where this is going. But the federal regulators of the monopolies are asleep at the switch, we can't even get network neutrality passed. The monopolies know how to play the lobbying game as well as how to slowly turn up the heat so the users aren't all outraged at once. But we can expect more abuses, more ads, more monitoring, more restrictions, more unwanted 'value adds' as time goes on.
| It's extremely bad. The fact that ISP monopolies
| are not regulated in favor of consumers is slowly
| going to destroy the openness that has made the
| web so successful.
This is a little over the top. Whether or not to use this to notify users of time-sensitive information could be a question posed at even a small ISP without such 'evil ambitions.'
It's probably more useful to discuss the pros/cons of this approach to notifying users than it is to decry over-arching problems with the entire industry. These (over-arching industry issuse) have been discussed ad nauseum, and action is more useful than discussion at this point (at least on technical forums such as this).
| But the federal regulators of the monopolies
| are asleep at the switch
I can't think of a worse way to message it. You don't inject your data into my private communications. No matter what.
The right way would be to ask the customer when they sign up for service what method they would like to receive service notices through. Phone, email, SMS, lettermail, twitter, Facebook, there's a million better ways than to modify my data.
Since you're a comcast employee, maybe go ask the guys running your SMTP/POP3/IMAP servers. I have faith that you guys can come up with some way to communicate with the people using them.
Rogers has been doing this for years in Canada already..
They use it to notify subscribers when they are approaching their bandwidth quota (75%) and then again when they hit 100%. You actually have to click a "I understand" button to have it not show up over and over.
Rogers also used to serve ads in place of an error message when a bad URL was requested. That was the final straw causing me to cancel my service with them and switch to Teksavvy.
I believe Comcast hijacked NXDOMAIN DNS replies and replaced them with their own IP address, causing every non-existant domain name to go to their search page you had to opt-out of.
I'm pretty sure you can forcefully opt out by using a DNS server that isn't run by scumbags, like 8.8.8.8 and 8.8.4.4 for Google Public DNS. I hear OpenDNS is similarly good.
OpenDNS does the same thing these people are complaining about -- they wrap the missing domain in something like a search page that has their logo and custom ads on it.
Originally I wrote a GreaseMonkey script to redirect me from their ad page to a Google search. At the time, it was better than nothing, but still not enough to keep me from dropping Rogers. YMMV, Last updated 2008: http://userscripts.org/scripts/show/30326
Afaik, Airtel still does it and people still put up with it. I think they do the usage % notification hijacking as well without understanding even a bit that that internet is a pipe and people use applications other than web browsers and protocols other than http.
Yea I was not sure. I left them after their data cap for "high speed" "unlimited" internet cap was 3 gb per month. I can put up with js injection but ridiculous data caps are something I can't live with.
That was reason #2. Going from a 60gb cap (which I'd often go over by about 20gb) to a 300gb cap actually saved me a lot of money because of overage charges.
The block of code they injected here was 7.9 KB (3.7 KB gzipped). jQuery is 93 KB (33 KB gzipped). So no, I don't think that would have been more elegant. Injecting anything into users' pages without permission is insane. Injecting a huge library like jQuery would be even more insane.
Wasted resources. The difference in size between the two (using the numbers from the above comment) is 85.1 KB. Now think of all the customers Comcast has and you will see quickly the difference it makes with a few KB.
You're missing the point. You're too focused on the code writing part. It's the extra unnecessary resources loaded from jQuery.
The difference in size is 85.1 KB (according to an above post). 85.1 KB * 100,000,000 (Just an example of the number of times it is loaded) = 7.92555511 terabytes of wastes resources.
Cox is doing something very similar. It's somewhat disconcerting to see JS like this ending up in pages, especially since they didn't get the URL right and a future version of this script could conceivably allow someone to serve malicious content to every Comcast subscriber, injected directly into your page.
So if you were proxying some other protocol over port 80, Comcast might just inject some JavaScript into the stream and corrupt your data?
I don't even like the thought that they're running some kind of hardware that makes this possible. They're sending packets impersonating a web server you actually want to talk to, pretending to be part of a response you requested?
"I don't even like the thought that they're running some kind of hardware that makes this possible"
It's called a proxy server. They're actually really common - many ISPs use them. Any hotspot that shows a log-in page in your browser will, and I know my University's internet goes through one.
Squid [1] is one of the most well-known, and it's open source.
Good luck fighting against a team of lawyers with virtually unlimited budget. If you're lucky you might get a cash settlement but they'll still be screwing everybody else with impunity.
The idea would be that websites would take action, not end users. (Otherwise, how would it be a copyright vio?) I think we can assume that if it was infringement, Google would have an interest and the pockets to go to battle.
IANAL, but I can't really see how it would be infringement, though.
This remains an untested field of copyright law, as far as I know. I've been waiting for literally over a decade for some test case on this matter to come up, and it never does. Perhaps by 2023.
Courts will generally refuse to take on manufactured cases. Their job is resolve real disputes.
A lower court would probably just throw the case out.
And if it didn't, the higher courts, which would set a widely binding precedent, would exercise their discretion simply not to hear the case. Yes: they get to pick and choose what appeals to hear.
Wonder how the folks back at Comcast HQ would feel if the rest of the internet started adding messages to their web browsing telling them this kind of thing is unsatisfactory? Hey, this content injection game is a game that we all can play.
This is the old "windows alert" nonsense. Everybody and their brother that touched the windows system thought the user would want a popup when their program did something. So the user experience was/is full of annoying popups, warnings, and information messages. Log onto a heavily-customized windows machine that hasn't been used in a month or two and it's like visiting Los Vegas. Good luck trying to get anything done.
Comcast. All kinds of other internet providers manage to communicate these things to their subscribers without this nonsense. Take a hint.
Interestingly it would be easy to write some code that detected THIS code. Get web developers to add it to their sites and make it show a message that comcast are charging them for traffic they're causing. And then link to the class action.
inject.ly isn't registered (yet) so let's presume some enterprising HN reader uses that.
As a web dev, all I need to do is <script src="//inject.ly/detect.js"></script> and it will detect this (and any future variant) ISP injected content.
Extra points for someone implementing this to have it optionally make a JS call to another function or inject a customisable HTML widget on the page.
Browser extension would require action from end-users, making educating them using it rather redundant. Doing detection in JS can easily be deployed on servers with minimal work needed, and can potentially reach a very wide audience.
Don't we just all need to put an appropriate JSON payload into '/e8f6b078-0f35-11de-85c5-efc5ef23aa1f/aupm/notify.do?dispatch=checkBulletin' on every web server we control?
This code is beyond awful - it fails to display, makes endless AJAX requests, and more; here are a few fun tidbits:
1. The code is not encapsulated in an IIFE, so it clobbers any global variables (like 'image_url') in the page, breaking any scripts relying on those variables.
2. The code spends an inordinate time checking if you're running Netscape Navigator 6.
3. Strangely, they include a whole bunch of code allowing the message to be dragged around the window (which is nice) but they don't allow it to be closed. Of course, it closes itself after making a single AJAX request into a black hole, so there's that. Bugs piled on top of each other make this entire message mostly harmless, if it weren't for the variable clobbering & bandwidth usage (see the next item...)
4. Upon load, checkBulletin() is immediately invoked. This does an AJAX call to '/e8f6b078-0f35-11de-85c5-efc5ef23aa1f/aupm/notify.do?dispatch=checkBulletin'. I assume this is to check if the bulletin has changed, to see if there are new messages, or maybe to check if the user has acknowledged the message yet. Unfortunately:
* This URL is relative, which means it will never actually reach its intended target (instead filling your web logs with this request)
* Upon xmlhttp.readystate=4 (request finished, successful or not, so this will change to 4 even on a 404 error), the comcast message is hidden. This means that the entire 'bandwidth exceeded' message will actually be hidden as soon as this request completes, which may be in <500ms, giving the user absolutely no time to see or acknowledge it.
* The author makes an attempt to not continue sending AJAX requests to this URL after a successful attempt, but botches it, so this request is actually sent indefinitely, every 5000ms, while every any page is open. This means every single tab on your system is popping AJAX requests every 5 seconds for the whole month that your account is nearing its quota. This likely brings you over quota pretty quickly if you leave your computer on all day.
That's right, this code causes every page served on your system to pop an AJAX request to the wrong URL every 5 seconds, as long as the tabs are open.
We can sit and argue all day whether or not it's ethical to display messages by injecting code into the DOM, but it is certainly unethical to write such awful javascript that clobbers global variables and drives up bandwidth costs by making AJAX requests to the wrong url every 5 seconds until the cows come home. Whoever wrote this script should be fired.
EDIT: Similarly, back in the dialup days, some ISPs would inject ads into their content. One way this was stopped was to argue that it was not legal for the ISP to charge you for data, then artificially inflate the size of that data by injecting ads. This script is doing just the same in a measurable way by causing these AJAX requests to be run every 5 seconds on every tab in your system.
They're all over the place. People just starting out. It could've been an intern fresh out of college. It could've been someone who just never graduated beyond copy-and-paste-from-StackOverflow. It could've been written by a person who never did web development before and was just told to make it work.
The little HN/Twitter/Reddit "awesome programmer" bubble is just that... a bubble. It's easy for us to forget that lots of people write lots of bad, untested code all day long. As much as it frustrates me, lots of people code who don't care about code - it's just their job.
> R3.1.1. Must Only Be Used for Critical Service Notifications
Additional Background: The system must only provide
critical notifications, rather than trivial notifications.
An example of a critical, non-trivial notification, which
is also the primary motivation of this system, is to advise
the user that their computer is infected with malware, that
their security is at severe risk and/or has already been
compromised, and that it is recommended that they take
immediate, corrective action NOW.
>They're all over the place. People just starting out. It could've been an intern fresh out of college. It could've been someone who just never graduated beyond copy-and-paste-from-StackOverflow. It could've been written by a person who never did web development before and was just told to make it work.
I'm an intern, just moving past S.O. copy-pasta jobs and generally get scared at what the hacker news crowd might say about my code... seeing this caliber of shit get pushed live by a major ISP is almost comical, if an admitted novice such as myself can see that it should be a sign as to the ineptitude of our current crop of ISPs.
Keep it up - keep moving up and learning more stuff. Be awesome. Don't worry too much about what other people think of your code, worry just enough that it pushes you to write better code. :)
While there's nothing * technically* wrong, it's platform specific and I think it would be better to use PHP's unlink[2] function. Also, sorry if this is wrong, I haven't looked at the regex but it seems your parsing YouTube URLs? Have you looked at oEmbed[3] - it may be an easier way to accomplish what your doing? You can use it with json_decode[4] to get an object.
The type of programmer who writes code like this never wonders whether their code could be better or not. So don't worry, just by being self-aware enough to ask the question you put yourself on a higher level.
Hrm, is there really a problem if the test data on my development server were to get dumped? It's not like those credentials or the accounts stored in the db carry over when this gets deployed to production, nor will the changes for production ever come close to my github.
Still very valuable things to be aware of in future situations where the above might not apply, thank you very much.
One thing I have learned over the years: It is easy to write "this is crap code" over a lot of production code I have seen. But making it better, writing consistently great code in the usual environment is much harder.
Don't let the macho attitude of HN infect you too much - a lot of people here (and elsewhere) are great in criticizing others.
+1. When you have whole pile of pretty bad code to maintain, it is very difficult to make the fixes significantly better within the time you have to make the fix. Usually significant improvements would require extensive refactoring which is feasible or sensible in surprisingly few cases.
Though, I have to admit, bitching about other peoples' code is fun.
This code is from a 10-year veteran "consultant," probably charging over $200/hour, brought on by the Global Services company hired by the Consulting Agency that Comcast brought in to assist in completing the critical time-sensitive project as quickly as possible.
It was also deemed a great success, and presentations were made about how effective it was, how smart the manager who hired the consulting agency is, and how skilled the global services contractors were who implemented it were, all only 2 weeks behind schedule—a new record for a project of this scope.
That manager got a promotion and is now VP of something or other. He sleeps like a baby and makes 100 times more than you.
They understand that "good code" is code that delivers a lot of value to the person who needs it. Why hate on them for that? You just sound jealous that they make more than you do.
That's the whole point. Code is not meant to serve the people who maintain it. Maintainability is only a concern once lack of such starts impacting your actual customers. If writing ugly code and fixing it up later is necessary in order to get shit out the door, why is that bad?
So because it satisfies the suits, he should reserve passing judgement? Try again; he is a programmer, not a suit. Hint: there exist many seperate but equally valid systems for judging worth/merit/quality.
Also, even for a suit, "Maintainability is only a concern once lack of such starts impacting your actual customers." is only true if by "actual customers" you mean shareholders. If you really want to get down to it and make an obnoxious out of place point, you can technically fuck over the customers all you want so long as doing so does not actually hurt the business (meaning: hurt the shareholders). Bonus points for figuring out how this could be done by a consulting company.
If you are a money-chasing robot, perhaps. Most businesses care at least a little about making a good product/satisfying their customers. That's good. Making life easy for your employees at the expense of your customers? That's bad.
Because if you want to be a software developer in the long term, you need to prefer the long term alternative in most cases.
A typical example is, "If we don't get something out the door, we'll be out of business. 'Shit' is something that can be shipped quickly, therefore we must ship 'shit'."
But companies that ship 'shit' generally go out of business anyway. Either their customers find it unappealing and leave, or ongoing maintenance quickly becomes so difficult and expensive that the product can not improve except by being rewritten under new management.
With something like a secure website (or script injected into arbitrary websites by a large ISP) the severity of the security vulnerabilities that tend to result from "shipping shit" often you only get one or two chances as a company.
"Good code" that delivers a lot of value and is high quality and maintainable is still better!
I'm not jealous. They don't make more than me. I said they make more than you. And I'm not hating—I'm just telling it exactly like it is, because I understand it, and it's insane, like the truth tends to be when you have huge amounts of power and money being controlled by puny incompetent humans.
This is exactly who wrote this code. I nearly accepted a job with one of Comcast's major consulting partners. My first hint should have been 2 technical interviews in which they were impressed that I used linux... and couldn't tell me a thing about what their day to day looked like. "Oh it's always different"
When I was issued my company laptop, the software had been installed by hand (OS and all). I offered to setup an imaging system for them... but the "IT guy" from the "IT consulting firm" wasnt exactly sure what that was and needed to find out who to get approval from first...
You know, there is nothing wrong in picking up code from stack overflow. If you are very efficient in picking up good, well written snippets that fit the style of the project and work without debugging and any time waste - more power to you.
Personally, I consider google search (and stack overflow) as an extension of my development environment and I'd recommend using it and melding your dev env with google search as much as possible. It really helps and speeds things up.
> * This URL is relative, which means it will never actually reach its intended target (instead filling your web logs with this request)
It likely doesn't matter that the URL is relative. It contains a GUID to be unlikely to resemble any real URL, and it's clear enough that they are capable of deep-packet-inspecting all of your web traffic from the way this is already used, so they likely hijack any request to this URL path within their network to capture its contents, and return a 200.
I don't have Comcast so I can't verify, but it would be interesting for somebody to check whether that URL is masked for all Comcast users.
> That's right, this code causes every page served on your system to pop an AJAX request to the wrong URL every 5 seconds, as long as the tabs are open.
I can only hope that they infinitely hang requests to their special URL in the case that user is under the quota so that this is not true. But if it is true, and they are not perfect about masking the URL (edit: it seems like people below on this thread have seen requests to this URL in their server logs), this could be construed as a DDOS attack by Comcast on every owner of an HTTP server via their own customers.
brokentone comments below that they've seen the urls in their production logs. I don't see it in any of mine but I'd be willing to bet that a company writing JS that bad would probably screw up the rest of the process too.
Surely a class action against Comcast is in order here? They're charging everyone for bandwidth they're not using.
There are a lot of things in this code that make me think that it was written by someone for whom JavaScript is not their main language - but probably the most glaring example is the use of `new Object()`. I've never seen anyone with more than 3 days JS experience use the Object constructor over a literal.
> This is code from someone who has no idea how to program
That's a quite strong assertion. What's wrong with your first example? I can think of very few criticisms (s isn't needed for example) but there's lots of things they did well:
- It follows the best practices for an OO constructor (doesn't return the object, just sets properties of `this`)
- All temporary variables are local. No global pollution (besides the "Browser" function itself, but because you're quoting it out of contect, I can't tell if even that's local or not)
- Degrades gracefully (everything is null) instead of picking a default incorrect choice
Sure, I would have written it differently, but so would everyone else here.
As for your second example, sure, it's not great, but I can sort of imagine some sleep-deprived developer coding up that to interop with some auto-generated DOM elements from an old PHP script left behind by a forgotten intern. We need more context here.
I thought that for a moment, but it isn't so. They do call the Browser() function as a constructor with 'new Browser()', so 'this' is the object it's constructing.
All your points are well taken, and a better analysis of the code by far than my hasty reaction.
So what was bothering me about the first example? Probably the repetition of the indexOf() tests, combined with one of the indexOf() tests being >= 1 and the rest >= 0.
But you're right, it's not nearly as bad as I made it out to be.
Since I've put my foot in my mouth, I guess I'll put my money there too and show how I might have done it. If I were doing UA detection at all, that is:
But that fails on one of your points, since it returns an object instead of setting properties of 'this'. It's also less flexible - what if one of the tests needed more than a simple string comparison? At least it's simpler?
So who am I to criticize? :-)
On the second example, it's not just that function - the entire web page is full of similar code. Here's another snippet:
addressCheckMsg="";
if(!type)
{
iLen = line1.value.length;
for(i=0; (i<4) && (i<iLen); i++)
{
var ch = line1.value.substring(0,i+3);
chUpper=ch.toUpperCase();
switch(chUpper)
{
case 'PO BOX':
addressCheckMsg += " Resident address can not be a P.O. Box.\n";
i=iLen;
break;
case 'P.O. BOX':
addressCheckMsg += " Resident address can not be a P.O. Box.\n";
i=iLen;
break;
case 'P. O. BOX':
addressCheckMsg += " Resident address can not be a P.O. Box.\n";
i=iLen;
break;
case 'P O BOX':
addressCheckMsg += " Resident address can not be a P.O. Box.\n";
i=iLen;
break;
case 'POB':
addressCheckMsg += " Resident address can not be a P.O. Box.\n";
i=iLen;
break;
default:
break;
}
}
}
Yikes. I'd better not say more or I'll start foaming again... :-)
Your first example has nice trickery in it but I prefer the original version. I think it is important to keep it simple.
Compressed code is often not the best way to do it; adding a few lines of verbosity can reduce the time it takes to understand the code to a fraction while sacrificing very little in terms of performance.
It's good that this discussion can actually be had on here civilly. Too often I see vitriol and pedantic disagreement on HN for no reason other than the swinging of the e-peen.
Comcast is such an incompetent company. I tried to sign up for service once and they charged me ten bucks to ship me two coax cables yet I was never able to get my service activated because I mistakenly thought my place was hooked up to cable when it wasn't and when I tried to call to correct this and schedule an installation I kept getting put on hold for a half hour before being given a message saying there was an error with their phone system and to call back. I mean seriously wtf.
after 2 weeks, 3 techs coming to my house,5 chat conversations and multiple phone calls I finally have service... I don't really like this legal monopoly for cable companies... I would switch to ATT but right now they are about twice the cost...
As a different point, my place was already pre-wired. I bought a cable modem from Best Buy, and plugged it in. It synced immediately. Then I went online and ordered service. They charged me $10 to send a self-install kit, but it wasn't needed I was actually online within minutes.
So sometimes their systems work...
I was very sad about switching from my other carrier (Sonic.net), but they ultimately couldn't deliver very much bandwidth. And Comcast was actually cheaper.
In my case, their published (non-promotional) rates were cheaper than the bonded DSL I was using. I really didn't want to switch, but I just couldn't justify the amount I was spending for the bandwidth I got.
That didn't work for me. I threatened to leave for CenturyLink DSL unless they could give me a better deal and the only thing the lady offered me was a triple play package for more than what I was already paying. So I had to switch over to DSL at 12/1 speeds.
The one good thing is that CenturyLink isn't part of that 6 strikes deal.
for companies like comcast the easy solution is chargeback as a SaaS owner i hate to promote the idea of chargeback - but seriously they sting bad and could really act as a good wake up call for companies like comcast.
I'm pretty sure Comcast aren't the only ones doing this. I had mobipcs for a while (when I just got new house, had to wait for DLS to get installed) and they injected js that tracked your browsing and replaced certain ads it found (as well as caused various errors because it wasn't written properly). I wouldn't be surprised if other companies did the same.
You don't have what? Comcast had a data cap nationwide (250GB/mo) even if you weren't aware of it. They temporarily stopped enforcing it outside of two test markets (Nashville and Tucson) where they're working out exactly what limits people will put up with. You wouldn't see this popup unless you live there and you've used over 225GB this month.
This is probably part of their "Web Notifcation System". They have a published RFC talking about how it works (RFC6108).
Using that system they can selectively notify customers. Like if they detect your system is infected with a virus. Or warn you your service will be discontinued if you don't pay your bill.
Look at all the work that went into that RFC. Unbelievable that they couldn't get a half-decent developer to verify that the notification is coded well enough to even show properly.
I agree. The entire concept is about trying to be less invasive in the web browsing experience (by adding a popup instead of redirect the entire web session) but that all falls apart because of crappy JavaScript.
My ISP has a similar system. Except it works like this: if your machine is detected to be sending spam, for instance, the next time you try to view a webpage you're served an information page that your PC is compromised, please fix it and click here to not see this page again next time. Your actual traffic isn't compromised, it's just redirected to let you know of a problem. I can't tell you what would happen if you got close to your data limit, since we don't have any.
HTTPS Everywhere[1]. Using SSL certificates helps prevent man-in-the-middle attacks[2], such as this. Comcast wouldn't be able to read any of your traffic and insert js without spoofing SSL certificates.
I've looked into the costs of VPNs on servers I own vs. a VPN service with unlimited bandwidth limits and the latter always wins. Setup time costs aside, VPN services usually have multiple regions you can connect to and are likely to have more reliable speeds.
This is, of course, if you trust these companies enough and [list of security implications].
A personal license of Umbrella by OpenDNS, which includes always-on laptop and phone VPN, is $20 a year currently. There are also features like Anycast that are not feasible to replicate on a personal server.
Because using Comcast using an HTTPS proxy to rewrite traffic means that your browser will expect data encrypted with Google's, Facebook's, Chase's, etc. certificate when it actually receives Comcast's proxy certificate. Every HTTPS site would prompt the user about an SSL certificate error.
They would either need the private key of the certificate holder (which they don't have), or a certificate signed by one of the roots installed on the system, which they also won't have.
I suppose the logical next step is that Comcast requires you to install a "Comcast Internet Helper" program that also installs a Comcast root certificate into the system so they can mitm anything.. But Firefox and Chrome would probably release updates mere hours later, blocking that cert from being used from those browsers.
Most people are going to click through any security warning because they just want to get to the site they wanted to go to. If Comcast does this, it would make EVERY SSL site display the warning, making it utterly meaningless.
Alternatively, it's not that outrageous to think that Comcast et al could get certs into the major browsers if they wanted to do so. It's not even implausible to think that at some point, browsers will be legally required to distribute ISP certs to allow for the "safety" of users.
If Comcast makes you install a custom application to keep your certs up, it won't matter if Fx and Chrome block each cert within hours, because Comcast can keep generating and pushing new ones out. And, as above, if the ISP is going to fiddle like this, the actual power held by browsers is greatly diminished -- users aren't going to use a browser that doesn't let them browse without nag screens on every page, even if it is "for their own good".
I think you're a bit out of touch as those browser warning pages have changed a lot the last few years. It's actually pretty hard to get through those warnings now in most of the browsers.
As I noted in a comment elsewhere in this thread, Comcast (or any other vendor) doesn't need to go through the work of getting certs into major browser. They just need to purchase and use a root signing certificate that works under the existing root CAs that are already in all the browsers.
This is part of why the trust model of the current CA system is fundamentally broken. We need to add a layer that can ensure that we are in fact using the SSL certificate that the site owner wants us to use.
> They would either need the private key of the certificate holder (which they don't have), or a certificate signed by one of the roots installed on the system, which they also won't have.
Actually, this is fairly common for firewalls and other edge devices to do and is one of the problems with the "trust" in the CA system. You can get a "signing certificate" from various legitimate sources (ex. http://www.sslshopper.com/article-trusted-root-signing-certi... ) that allows your product/service to terminate SSL connections and then recreate a SSL connection. The user still sees their "lock" icon and thinks they have a secure https connection to their original site, when in fact they don't.
They do have a SSL connection to their site using a certificate - it's just NOT the certificate that the original site issued. This is why many of us are looking to protocols like DANE that uses DNSSEC to add a layer of integrity protection so that you can know that you are using the correct SSL certificate. (See http://www.internetsociety.org/deploy360/resources/dane/ )
Note that no new certificates need to be added to browsers. The signing certificates work with the existing root certificates that are already in browsers.
Ouch, I wasn't aware that CAs issued this type of "root" certificates. Is this very common, or will only some CAs do it? If the latter, I'll definitely remove them from my computers list of trusted CAs..
Edit: read the DANE article, seems very sensible and simple to implement that the server specifies valid certificates.
They're probably just desperate because for some strange reason, people don't seem to be getting the alerts sent to their @comcast.net email addresses...
Hey man, this thread really took off! Nice writeup here, if I saw that, I would have submitted that instead. I submitted this right before I left work, after noticing the requests on my server and a quick Google search (on the UUID) turned up your gist and not much else. As a web server, I was kind of trying to start some discussion to see if I was alone in seeing this and didn't expect it to get to #2.
Interesting side effect of not serving the entire blog post on the blog itself - the code in your posts won't be indexed by Google on your site, only on gist.github.hom?
I had just moved my blog to a new host. I had done an import of my blog using the Wordpress plugin instead of just exporting the entire database to help clean things up.
I forgot to install the gist plugin so my blog post no longer contained the code. I also had 3 different domains serving the same blog due to a misconfiguration with Nginx which caused my blog to take a temporary hit on Google.
I've since addressed those things so hopefully those will make my post actually appear in a google search.
Is there a downside about notifying the FBI of this? I encouraged the op to do so in another post, encouraged. If this was happening on a 56k modem over a phone line it would clearly be wire tapping.
I do not see any downsides to contact the FBI about the matter, if you think of any please let me know.
Has this been confirmed to still be happening? The guys blog post[1] states that this was on Nov 20th 2012. Anyone currently using a comcast account want to put down their pitchfork for a second and help verify this?
Comcast only enforces data caps in two cities right now, so your testing pool is much more limited than simply anyone using Comcast. Ryan (the author of the blog post) lives in one of those two cities. A potential tester would have to be in either the Nashville or Tucson area, and have used over 90% of their bandwidth cap for the month.
Hi awj, I'm the author of the blog post. As dangrossman said, Comcast only enforces the limit in 2 cities. I live in the Nashville area, so I'm affected. They just doubled my 50Mbps connection to 100Mbps so I will go over my limit this month as I have 2 more grace periods left. If it happens again I'll update my blog post.
Are there down sides to contacting the FBI about this? They in part exist to document and keep track of potential crimes(potentially correlate them over long time frames that may not be worth while to keep track of for an individual but can add great benefit to society at large when the burden and information is centralized.) This seems like it would fall under their definition of internet crime found on:http://www.ic3.gov/faq/default.aspx.
If there are not major down sides please file a complaint with the FBI, I believe the url is:http://www.ic3.gov/default.aspx.
I encourage you to explain
* your evidence that when accessing various websites they appear to be tampered with between the server and your computer.
* Your worry that it impacts your bill with Comcast as it seems to be eating up you bandwidth. An estimate of the amount of money being eaten up if you have reason to suspect it is a city wide occurrence how much money is lost for everyone across the city?
* If you have packet logs of these occurrences I encourage you to include them.
* Unless you have hard evidence that points to Comcast that is doing the tampering I would not accuse any party of responsibility.
* If you have concerned friends who can independently verify similar conditions, it would probably be valuable to have them file similar complaints, referencing each other where applicable.
287 comments
[ 2.8 ms ] story [ 244 ms ] threadI don't really understand the point of this, either. Couldn't they starting redirecting users to a static page somewhere if there were a real need for a "critical and time sensitive" alert? If the supposed alerts aren't critical enough to justify doing that, use email, IM, RSS or twitter, or even build a custom notification notification app.
This seems bad, but the warning (exceeding your bandwidth quota) seems valuable. I can't think of another, better way to message this.
I wonder isn't there any law in US that forbids carriers from fiddling with messages?
They can do pretty much whatever they want to.
Injecting into a webpage is unacceptable.
1. They could email you. 2. They could send you a SMS. 3. They could let you view your bandwidth usage by logging into their site. 4. They could provide an application (desktop or mobile) to keep track of your bandwidth and alert you at certain points.
They still send interstitial content warning me that I've exceeded my fair-use limit. It's a bit annoying because I very carefully checked what the limits were before I signed up.
What's worse is that they use weird, broken, IP addresses and horrible proxies for image mangling.
EDIT: Here's a pastebin.
(http://pastebin.com/k6ddD0sJ)
EDIT: Here's a Security \\\stack Exchange question about it: (http://security.stackexchange.com/questions/9368/mobile-carr...)
But yeah, if you have service with Comcast they have your home phone, email addresses, and physical address. They can get in touch with you every way that every company that CAN'T read all of your internet traffic already gets in touch with you.
The method they've chosen is terrible for at least the following reasons: - The alert will not work on many platforms & devices. - The alert may not reach the account owner. - The alert will not work on SSL traffic. - There is no record that the customer saw the alert (contrast with phone call) - There are serious privacy issues involved in parsing user's web traffic.
If you look at what Comcast does on the TV side, things like adding ads to the guide so it's barely usable, you can see where this is going. But the federal regulators of the monopolies are asleep at the switch, we can't even get network neutrality passed. The monopolies know how to play the lobbying game as well as how to slowly turn up the heat so the users aren't all outraged at once. But we can expect more abuses, more ads, more monitoring, more restrictions, more unwanted 'value adds' as time goes on.
It's probably more useful to discuss the pros/cons of this approach to notifying users than it is to decry over-arching problems with the entire industry. These (over-arching industry issuse) have been discussed ad nauseum, and action is more useful than discussion at this point (at least on technical forums such as this).
Look up regulatory capture.The right way would be to ask the customer when they sign up for service what method they would like to receive service notices through. Phone, email, SMS, lettermail, twitter, Facebook, there's a million better ways than to modify my data.
Since you're a comcast employee, maybe go ask the guys running your SMTP/POP3/IMAP servers. I have faith that you guys can come up with some way to communicate with the people using them.
Rogers has been doing this for years in Canada already..
They use it to notify subscribers when they are approaching their bandwidth quota (75%) and then again when they hit 100%. You actually have to click a "I understand" button to have it not show up over and over.
The whole document.write block 27-51 (possibly the CSS-block too, but I'm not sure about this) could be written far more elegant in jQuery.
But the real saving is that "drag and drop" code - jQuery would abstract all that isIE/isNS crap from them.
...unless you're still being sarcastic?
a 1 page script written for comcast does not demand jQuery.
That's not to say it would have improved it.
The difference in size is 85.1 KB (according to an above post). 85.1 KB * 100,000,000 (Just an example of the number of times it is loaded) = 7.92555511 terabytes of wastes resources.
http://www.google.com/patents/US20110264729
Which I can tell you for certain that they don't own. Bastards.
I don't even like the thought that they're running some kind of hardware that makes this possible. They're sending packets impersonating a web server you actually want to talk to, pretending to be part of a response you requested?
Just because you can do this doesn't mean you should (i will stay away from comcast xfinity).
i have had these types of issues a while back at coffee shops that try to inject ads, it was breaking my XML.
It's called a proxy server. They're actually really common - many ISPs use them. Any hotspot that shows a log-in page in your browser will, and I know my University's internet goes through one.
Squid [1] is one of the most well-known, and it's open source.
1. http://www.squid-cache.org
IANAL, but I can't really see how it would be infringement, though.
1) building a webpage where you own the copright
2) Have someone in one of the cities where this is happening browse to your page.
3) Copyright violated, and you get to be the test case!
A lower court would probably just throw the case out.
And if it didn't, the higher courts, which would set a widely binding precedent, would exercise their discretion simply not to hear the case. Yes: they get to pick and choose what appeals to hear.
This is the old "windows alert" nonsense. Everybody and their brother that touched the windows system thought the user would want a popup when their program did something. So the user experience was/is full of annoying popups, warnings, and information messages. Log onto a heavily-customized windows machine that hasn't been used in a month or two and it's like visiting Los Vegas. Good luck trying to get anything done.
Comcast. All kinds of other internet providers manage to communicate these things to their subscribers without this nonsense. Take a hint.
inject.ly isn't registered (yet) so let's presume some enterprising HN reader uses that.
As a web dev, all I need to do is <script src="//inject.ly/detect.js"></script> and it will detect this (and any future variant) ISP injected content.
Extra points for someone implementing this to have it optionally make a JS call to another function or inject a customisable HTML widget on the page.
;-)
If I'm not wrong, this is implementation of JSONP to avoid cross-domain AJAX request block, right?
1. The code is not encapsulated in an IIFE, so it clobbers any global variables (like 'image_url') in the page, breaking any scripts relying on those variables.
2. The code spends an inordinate time checking if you're running Netscape Navigator 6.
3. Strangely, they include a whole bunch of code allowing the message to be dragged around the window (which is nice) but they don't allow it to be closed. Of course, it closes itself after making a single AJAX request into a black hole, so there's that. Bugs piled on top of each other make this entire message mostly harmless, if it weren't for the variable clobbering & bandwidth usage (see the next item...)
4. Upon load, checkBulletin() is immediately invoked. This does an AJAX call to '/e8f6b078-0f35-11de-85c5-efc5ef23aa1f/aupm/notify.do?dispatch=checkBulletin'. I assume this is to check if the bulletin has changed, to see if there are new messages, or maybe to check if the user has acknowledged the message yet. Unfortunately:
* This URL is relative, which means it will never actually reach its intended target (instead filling your web logs with this request)
* Upon xmlhttp.readystate=4 (request finished, successful or not, so this will change to 4 even on a 404 error), the comcast message is hidden. This means that the entire 'bandwidth exceeded' message will actually be hidden as soon as this request completes, which may be in <500ms, giving the user absolutely no time to see or acknowledge it.
* The author makes an attempt to not continue sending AJAX requests to this URL after a successful attempt, but botches it, so this request is actually sent indefinitely, every 5000ms, while every any page is open. This means every single tab on your system is popping AJAX requests every 5 seconds for the whole month that your account is nearing its quota. This likely brings you over quota pretty quickly if you leave your computer on all day.
That's right, this code causes every page served on your system to pop an AJAX request to the wrong URL every 5 seconds, as long as the tabs are open.
We can sit and argue all day whether or not it's ethical to display messages by injecting code into the DOM, but it is certainly unethical to write such awful javascript that clobbers global variables and drives up bandwidth costs by making AJAX requests to the wrong url every 5 seconds until the cows come home. Whoever wrote this script should be fired.
EDIT: Similarly, back in the dialup days, some ISPs would inject ads into their content. One way this was stopped was to argue that it was not legal for the ISP to charge you for data, then artificially inflate the size of that data by injecting ads. This script is doing just the same in a measurable way by causing these AJAX requests to be run every 5 seconds on every tab in your system.
Not to mention the potential GPL violation for cut-n-pasting brainjar code.
http://www.brainjar.com/terms.asp
The little HN/Twitter/Reddit "awesome programmer" bubble is just that... a bubble. It's easy for us to forget that lots of people write lots of bad, untested code all day long. As much as it frustrates me, lots of people code who don't care about code - it's just their job.
[1] http://tools.ietf.org/html/rfc6108
> R3.1.1. Must Only Be Used for Critical Service Notifications Additional Background: The system must only provide critical notifications, rather than trivial notifications. An example of a critical, non-trivial notification, which is also the primary motivation of this system, is to advise the user that their computer is infected with malware, that their security is at severe risk and/or has already been compromised, and that it is recommended that they take immediate, corrective action NOW.
So much for that.
I'm an intern, just moving past S.O. copy-pasta jobs and generally get scared at what the hacker news crowd might say about my code... seeing this caliber of shit get pushed live by a major ISP is almost comical, if an admitted novice such as myself can see that it should be a sign as to the ineptitude of our current crop of ISPs.
One thing I can say is don't use exec[1] if you can avoid it:
While there's nothing * technically* wrong, it's platform specific and I think it would be better to use PHP's unlink[2] function. Also, sorry if this is wrong, I haven't looked at the regex but it seems your parsing YouTube URLs? Have you looked at oEmbed[3] - it may be an easier way to accomplish what your doing? You can use it with json_decode[4] to get an object.[1] https://github.com/Machtap/GiffyTube/blob/master/download.ph...
[2] http://php.net/manual/en/function.unlink.php
[3] http://apiblog.youtube.com/2009/10/oembed-support.html
[4] http://php.net/manual/en/function.json-decode.php
https://github.com/Machtap/_ctv/blob/master/_www/model/commo...
- keep config variables in a separate file that is in your .gitignore and won't get pushed to github.
- keep config file outside of any web accessible directory in case the file renders in plaintext for some reason.
Regardless of db only accepting local connections - an attacker is one step closer to dumping the db.
Still very valuable things to be aware of in future situations where the above might not apply, thank you very much.
Don't let the macho attitude of HN infect you too much - a lot of people here (and elsewhere) are great in criticizing others.
Though, I have to admit, bitching about other peoples' code is fun.
Ha, you give them too much credit.
This code is from a 10-year veteran "consultant," probably charging over $200/hour, brought on by the Global Services company hired by the Consulting Agency that Comcast brought in to assist in completing the critical time-sensitive project as quickly as possible.
It was also deemed a great success, and presentations were made about how effective it was, how smart the manager who hired the consulting agency is, and how skilled the global services contractors were who implemented it were, all only 2 weeks behind schedule—a new record for a project of this scope.
That manager got a promotion and is now VP of something or other. He sleeps like a baby and makes 100 times more than you.
Also, even for a suit, "Maintainability is only a concern once lack of such starts impacting your actual customers." is only true if by "actual customers" you mean shareholders. If you really want to get down to it and make an obnoxious out of place point, you can technically fuck over the customers all you want so long as doing so does not actually hurt the business (meaning: hurt the shareholders). Bonus points for figuring out how this could be done by a consulting company.
A typical example is, "If we don't get something out the door, we'll be out of business. 'Shit' is something that can be shipped quickly, therefore we must ship 'shit'."
But companies that ship 'shit' generally go out of business anyway. Either their customers find it unappealing and leave, or ongoing maintenance quickly becomes so difficult and expensive that the product can not improve except by being rewritten under new management.
With something like a secure website (or script injected into arbitrary websites by a large ISP) the severity of the security vulnerabilities that tend to result from "shipping shit" often you only get one or two chances as a company.
I'm not jealous. They don't make more than me. I said they make more than you. And I'm not hating—I'm just telling it exactly like it is, because I understand it, and it's insane, like the truth tends to be when you have huge amounts of power and money being controlled by puny incompetent humans.
When I was issued my company laptop, the software had been installed by hand (OS and all). I offered to setup an imaging system for them... but the "IT guy" from the "IT consulting firm" wasnt exactly sure what that was and needed to find out who to get approval from first...
Personally, I consider google search (and stack overflow) as an extension of my development environment and I'd recommend using it and melding your dev env with google search as much as possible. It really helps and speeds things up.
I've come across lots of situations where the accepted answer isn't the best answer.
So I'm talking about knowing vs. cargo-culting.
Is it because we don't like Comcast?
It's a rockstar developer thing. You wouldn't understand.
It likely doesn't matter that the URL is relative. It contains a GUID to be unlikely to resemble any real URL, and it's clear enough that they are capable of deep-packet-inspecting all of your web traffic from the way this is already used, so they likely hijack any request to this URL path within their network to capture its contents, and return a 200.
I don't have Comcast so I can't verify, but it would be interesting for somebody to check whether that URL is masked for all Comcast users.
> That's right, this code causes every page served on your system to pop an AJAX request to the wrong URL every 5 seconds, as long as the tabs are open.
I can only hope that they infinitely hang requests to their special URL in the case that user is under the quota so that this is not true. But if it is true, and they are not perfect about masking the URL (edit: it seems like people below on this thread have seen requests to this URL in their server logs), this could be construed as a DDOS attack by Comcast on every owner of an HTTP server via their own customers.
Surely a class action against Comcast is in order here? They're charging everyone for bandwidth they're not using.
That's a quite strong assertion. What's wrong with your first example? I can think of very few criticisms (s isn't needed for example) but there's lots of things they did well:
- It follows the best practices for an OO constructor (doesn't return the object, just sets properties of `this`)
- All temporary variables are local. No global pollution (besides the "Browser" function itself, but because you're quoting it out of contect, I can't tell if even that's local or not)
- Degrades gracefully (everything is null) instead of picking a default incorrect choice
Sure, I would have written it differently, but so would everyone else here.
As for your second example, sure, it's not great, but I can sort of imagine some sleep-deprived developer coding up that to interop with some auto-generated DOM elements from an old PHP script left behind by a forgotten intern. We need more context here.
All your points are well taken, and a better analysis of the code by far than my hasty reaction.
So what was bothering me about the first example? Probably the repetition of the indexOf() tests, combined with one of the indexOf() tests being >= 1 and the rest >= 0.
But you're right, it's not nearly as bad as I made it out to be.
Since I've put my foot in my mouth, I guess I'll put my money there too and show how I might have done it. If I were doing UA detection at all, that is:
But that fails on one of your points, since it returns an object instead of setting properties of 'this'. It's also less flexible - what if one of the tests needed more than a simple string comparison? At least it's simpler?So who am I to criticize? :-)
On the second example, it's not just that function - the entire web page is full of similar code. Here's another snippet:
Yikes. I'd better not say more or I'll start foaming again... :-)Compressed code is often not the best way to do it; adding a few lines of verbosity can reduce the time it takes to understand the code to a fraction while sacrificing very little in terms of performance.
So sometimes their systems work...
I was very sad about switching from my other carrier (Sonic.net), but they ultimately couldn't deliver very much bandwidth. And Comcast was actually cheaper.
At least for the first 6/12 months. Then you get to haggle and threaten disconnection for a day, then you are good for another 6/12 months.
The one good thing is that CenturyLink isn't part of that 6 strikes deal.
I'm guessing this is their clever way of reminding you to pay the bill when you're late?
Pay your bill or they'll stuff ugly JavaScript in your browser, you've been warned!
You have reached 90% of your <b>monthly data usage allowance</b>.
Using that system they can selectively notify customers. Like if they detect your system is infected with a virus. Or warn you your service will be discontinued if you don't pay your bill.
http://tools.ietf.org/html/rfc6108
[1] https://chrome.google.com/webstore/detail/https-everywhere/g...
[2] http://security.stackexchange.com/questions/8145/does-https-...
Shame on you, Comcast.
This is, of course, if you trust these companies enough and [list of security implications].
(disclosure - I start working with OpenDNS soon).
Your bank, for another. Indeed there are far too many parties with an interest in keeping https secure, that you needn't worry about it.
But that ridiculous, right? Since everyone verifies SSL cert signatures...
I suppose the logical next step is that Comcast requires you to install a "Comcast Internet Helper" program that also installs a Comcast root certificate into the system so they can mitm anything.. But Firefox and Chrome would probably release updates mere hours later, blocking that cert from being used from those browsers.
Alternatively, it's not that outrageous to think that Comcast et al could get certs into the major browsers if they wanted to do so. It's not even implausible to think that at some point, browsers will be legally required to distribute ISP certs to allow for the "safety" of users.
If Comcast makes you install a custom application to keep your certs up, it won't matter if Fx and Chrome block each cert within hours, because Comcast can keep generating and pushing new ones out. And, as above, if the ISP is going to fiddle like this, the actual power held by browsers is greatly diminished -- users aren't going to use a browser that doesn't let them browse without nag screens on every page, even if it is "for their own good".
This is part of why the trust model of the current CA system is fundamentally broken. We need to add a layer that can ensure that we are in fact using the SSL certificate that the site owner wants us to use.
There are multiple solutions being proposed out there to add this trust layer. I am a strong advocate of DANE ( http://www.internetsociety.org/deploy360/resources/dane/ ) but there are others out there, too.
There was a good talk about this at Black Hat USA 2011 on "SSL and the Future Of Authenticity" at: http://www.youtube.com/watch?v=Z7Wl2FW2TcA
Actually, this is fairly common for firewalls and other edge devices to do and is one of the problems with the "trust" in the CA system. You can get a "signing certificate" from various legitimate sources (ex. http://www.sslshopper.com/article-trusted-root-signing-certi... ) that allows your product/service to terminate SSL connections and then recreate a SSL connection. The user still sees their "lock" icon and thinks they have a secure https connection to their original site, when in fact they don't.
They do have a SSL connection to their site using a certificate - it's just NOT the certificate that the original site issued. This is why many of us are looking to protocols like DANE that uses DNSSEC to add a layer of integrity protection so that you can know that you are using the correct SSL certificate. (See http://www.internetsociety.org/deploy360/resources/dane/ )
Note that no new certificates need to be added to browsers. The signing certificates work with the existing root certificates that are already in browsers.
Edit: read the DANE article, seems very sensible and simple to implement that the server specifies valid certificates.
Here's my writeup on it for whoever is interested
http://blog.ryankearney.com/2013/01/comcast-caught-intercept...
I forgot to install the gist plugin so my blog post no longer contained the code. I also had 3 different domains serving the same blog due to a misconfiguration with Nginx which caused my blog to take a temporary hit on Google.
I've since addressed those things so hopefully those will make my post actually appear in a google search.
I do not see any downsides to contact the FBI about the matter, if you think of any please let me know.
[1] http://blog.ryankearney.com/2013/01/comcast-caught-intercept...
If there are not major down sides please file a complaint with the FBI, I believe the url is:http://www.ic3.gov/default.aspx.
I encourage you to explain
* your evidence that when accessing various websites they appear to be tampered with between the server and your computer.
* Your worry that it impacts your bill with Comcast as it seems to be eating up you bandwidth. An estimate of the amount of money being eaten up if you have reason to suspect it is a city wide occurrence how much money is lost for everyone across the city?
* If you have packet logs of these occurrences I encourage you to include them.
* Unless you have hard evidence that points to Comcast that is doing the tampering I would not accuse any party of responsibility.
* If you have concerned friends who can independently verify similar conditions, it would probably be valuable to have them file similar complaints, referencing each other where applicable.