This raises the question: Is port scanning without consent a violation of the CFAA? Either it is legal, and researchers should face no repercussions for doing so, or it isn't and eBay is non-compliant with CFAA. I recall hearing about someone either being arrested or convicted due to port scanning a courthouse, but it was many years ago and I can't find the case with a cursory Google search.
I have to wonder what value eBay would get from port scanning its customers. Is it part of an attempt to detect bots/attackers? Is malware running on their server trying to determine if the client is likely vulnerable to some propagation method?
IANAL, but more likely it depends on intent and context. So shodan.io is okay because it’s not explicitly malicious, and they have clear paths to contact them if you suspect abuse. Whereas, if you’re suspected of hacking a website, the fact that you port scanned it a week prior to password spraying it might serve as evidence against you. That is, it seems unlikely anyone would be prosecuted for port scanning alone, but it could be an act that demonstrates intent of a later action.
One time, I port scanned my public IP (of my ISP) from an EC2 box, and I got an email from EC2 saying they received an abuse complaint from the ISP for port scanning activity.
What's Shodan.io's legitimate use? Sounds like the "torrents can be used for legitimate content" type argument where in reality you a rounding error the use is not lawful??
If I were a serious baddie, I'd be afraid of using Shodan. Who knows who has what logging on that, and what honeypots may have been seeded into it for just such an occasion? It's not that hard to get that information yourself, from sources you control yourself.
Legitimate usage from researchers and people reading about infrastructure they have the right to do security testing on may be a larger percentage than you think.
There are plenty of legitimate uses of port scanning, and specifically, a port scanning database like Shodan. For example:
- Monitoring your own network or that of your clients for exposed ports
- Researching Internet topology, or performing aggregate queries like “how many nginx servers are connected to the Internet”
Can you use it maliciously? Yes. But, most of the time, if you have a target it would make more sense to do the port scan yourself. And if you’re just dragnet searching for vulnerabilities, most you find will probably already have been exploited. Sites like shodan are good for the overall health of the web because they force website owners to maintain security posture. If you know that foregoing a wordpress upgrade means you’re one script kiddy with a shodan account away from getting hacked, you’re going to keep your site up to date. This saves you from script kiddies, but also from the more sophisticated hackers who would run a port scan themselves anyway.
>There are plenty of legitimate uses of port scanning, and specifically, a port scanning database like Shodan. //
Any legitimate security service is going to be doing there own scans, surely.
Statistics, yes, but I can't see those stats being especially good. You could probably get equally good nGinx data from netcraft, who IIUC get the data from http responses banners on :80 :443.
I'm not sure I buy the "security posture" line, isn't it circular. Tools to help crack your site are good because it means to have to have counter-measures to combat tools for cracking your site?
Only legitimate use of port scanning for me has been testing access to my own/clients computers, I feel. That's not too say I've not used it for illegitimate things ...
Shodan is used by most of the Fortune 100 companies for a variety of use cases. Here are the most common ones:
1. External network monitoring: know what you have connected to the Internet and get notified if anything changes unexpectedly. This has actually gotten significantly more challenging with services deployed to the cloud where your IT department might not even know which IPs to keep track of.
2. 3rd-party risk assessment: understand the security exposure of your partners, vendors, supply chain or other 3rd-parties. For example, lets say you're an insurance company that wants to provide cyber insurance. Shodan data can help you understand what sort of risk you'd be taking on. The data has also been used in M&A as part of due diligence to get a metric on the security of the IT department of the company they're thinking of acquiring.
3. Market intelligence: basically Netcraft on steroids. Shodan doesn't just have web information but also for many other protocols. This information is used by hedge funds and vendors to understand which products are purchased and deployed to the Internet. The data is skewed due to the nature of public IPs but there are still things you can do.
4. Policy impact: get a measure for how policies at the country-level are impacting Internet connectivity. For example, the OECD used Shodan to get a measure of Internet-connectivity per capita.
5. Fraud detection: is your customer trying to make a purchase from a machine that's been compromised? Or running a a VPN/ proxy? Shodan is used in transactional fraud detection to flag suspicious payments.
IANAL but this type of websocket port scan seems inherently different from what Shodan does.
Shodan is outside your network's firewall, therefore only able to access services you've exposed to the wider web.
If I understand the article, the websocket scan eBay is doing is trying to connect to local listeners on your laptop, behind your network's firewall and possibly even behind your laptop's firewall.
This is such an obvious consequence of web sockets that I wonder how anyone could have entertained the idea long enough to sober up and write the code. This is worse than letting a web page script have access to the clipboard, record mouse movements, and similar information leaks, because instead of just stealing information, now a web page can actively compromise any host on your network.
It does not, however, sound like an attacker can establish arbitrary TCP connections (at least using the technique from the article). Instead, the attacker can determine if something is listening on a port because it will take a different amount of time to negotiate/drop a connection to a port when there is a listener than when there is not a listener.
In other words, this sounds like a variant of a timing attack. As such, presumably, this particular avenue of attack can be mitigated by the browser vendor inserting a delay s.t. no information can be gleaned from how long it takes to negotiate/drop a websocket connection.
EDIT: I also wonder if it would be possible to do a similar port scan using the timing of XHR requests to localhost (e.g. http://localhost:[port]).
> It does not, however, sound like an attacker can establish arbitrary TCP connections
Maybe not, but what if the ports you have open actually are HTTP servers for development purposes? In that case wouldn't a website be able to crawl your unreleased work, and/or mess with what you're doing, with requests seemingly "out of nowhere"?
No, I think it was probably close to a decade ago, but I likely am misremembering some of the details. Could've been a police department, but I'm not sure.
That one you linked is a messed up case. There is a phenomenal podcast that interviews those guys and walks through their engagement. https://darknetdiaries.com/episode/59/
nmap's "Legal Issues" section states that the guy went on to start a successful digital forensics company, after spending years crushed under 6-figure legal fees.
Someone's never caught a case. You may not think this is true but the way the legal system works is everything is illegal. Then when they need to get you you are already guilty. Everyone else they just ignore. Crazy, right?
LMAO a downvote in 3 seconds. Good old hacker news. Well, downvotes don't make you right. I'm trying to open your eyes and you downvote me. I guess that is to be expected.
Not quite the same thing. A legal system could use presumed guilt (defaulting to assuming an accusation is true) while still having a 'blacklist' approach to which actions are punishable.
Do you know how many laws there are? Not to mention common law, which is law established by previous court decisions on matters that have never been covered by any statute?
Homogeneous idiocracy makes it boring. Knowing conversations that aren't the right conversations won't be allowed is what makes it boring. Citing the site fucking guidlines ...makes ...it ...bor
When did networking support we often had old code + lower quality equipment that could / would crash if you used off the shelf security software that would go out and scan and then try all sorts of things and then generate a report.
I'd say 90% of the time the powers that be at the company had no idea someone was running that software, or that it was still running at their company, and then someone moved a firewall and the system was exposed to more than intended. Then they'd turn it of ... and find another similar tool running somewhere else.
It could be a simple as a test or security system run amok.
> I'd say 90% of the time the powers that be at the company had no idea someone was running that software, or that it was still running at their company, and then someone moved a firewall and the system was exposed to more than intended. Then they'd turn it of ... and find another similar tool running somewhere else.
This demonstrates the absurdity of the CFAA more than anything else. Sorry for sounding like a broken record but the CFAA is not salvageable and MUST be repealed.
> I have to wonder what value eBay would get from port scanning its customers.
From the article:
> Looking at the list of ports they are scanning, they are looking for VNC services being run on the host, which is the same thing that was reported for bank sites.
> VNC is sometimes run as part of bot nets or viruses as a way to remotely log into a users computer. There are several malware services that leverage VNC for these purposes.
They're almost certainly doing it as part of a heuristic to detect bots. Hence the VNC / RDP ports. I would assume it's quite common for bots to have those ports open so they can be monitored
In this case it's a script running on your own web browser that's scanning localhost. You are effectively scanning yourself. It's in a different realm from external scans.
It's probably part of their fraud detection and mitigation strategy. Combined with other info about your transactions it could help raise a flag about changes.
As for the CFAA that's seemingly down to how aggressive the prosecutor is feeling about your case. I don't think it should be there's no real access happening and unless it's extremely aggressive and degrades network connectivity it's hard to argue there's any real damage done.
Almost certainly not. Commercial unauthorized port scans are utterly routine. There are well-known companies premised on it.
You can get to the same answer axiomatically from the text and case history of CFAA (a port scan literally can't grant you the access a CFAA claim needs to prove you intended), but that's obviously treacherous for non-experts to do; instead, the empirical demonstration should be conclusive here.
I don't know why this scan is occurring, but fingerprinting is the most obvious guess, and intrusive fingerprinting performed by real companies is usually about ATO prevention, which means they're not going to tell you any more about it (ATO defense is an arms race).
> Furthermore, when I installed and ran a VNC server, I didn't detect any difference in site behavior - so why is it looking for it?
I think behind the scenes they keep log of some sort of fraud risk, e.g. geoip different from billing country, suddenly a new operating system, vnc/teamviewer running would probably flag your account (even for benign purposes, e.g. you can get your money back or purchase cancelled if that info can prove your transaction was actually unauthorized).
I worked on a ecommerce where the previous developers implemented a rudimentary "score" system like that so that suspicious orders would be put in queue for phone verification (this was pre gdpr)
Don't try to understand this with metaphors. It's a trap. Port scanning isn't enough like anything in the real world for the analogies to apply.
I'd strengthen pfundstein's claim; port scanning intrinsically works by trying to open connections. That isn't enough "like" any particular physical thing to make it a correct analogy; it isn't knocking, or walking in, or opening, or anything else. But one thing we can say, without using analogies, is that it is definitely an active effort, an action deliberately taken, not something passive like "looking" is in the real world. That is not on its own proof that it is wrong... I am merely saying, it is certainly active, not passive.
I think your final objection fails by bad analogy too: one has to _actively_ direct ones gaze, in general, if one is to notice a type of lock or other security arrangements.
I'm not analogizing to "actively looking"; I am saying, it is an active action. You can tell it's active because if they don't make deliberate decisions to write code that performs this scan, no scan will happen. They have had meetings about this functionality, and implemented it, and tested it, and management has signed off on it, and in a place like eBay quite likely their legal department has signed off on it. It is an action they have taken, with deliberation and intention; it is not a thing that just suddenly started happening to them one day, like, Firefox shipping a new browser that has a new default font or something.
I'm referring to the literal, probably-hundreds-of-person-hours actions taken to create this functionality. This is relevant to both ethical and legal analyses. No analogy.
In this case, it's asking a clueless kid that lives in this house to try to open the selected door, and you guess whether the door was open or closed by the time after he returns.
I think it comes under the Criminal Attempts Act 1981. But it might come down to whether you gather information, if you test a port, intending later - if successful - to attempt an exploit, then that seems like it could be unlawful (under the Law of England & Wales). Keeping a record of ports found with services could be sufficient to demonstrate intent.
[Just to be clear, I'm not endorsing/condemning the law here.]
It is not fine. Just like burglars looking at a house is not fine (still nothing to do with being legal or not). Intent defines if something is OK or not. Just like killing is always illegal but when you do it to save someone else it is hmm less bad, you see intent...
What are the intents of people doing scan? I don't see any legitimate reason for scanning random people computers or servers that are not yours.
Was thinking the same, maybe creating a service account for the browsers, I believe they do need some level of elevation but then using the firewall rules disabling everything expect http/s ports.. Websockets and others might be an issue, might need to be updated on an adhoc basis though and shouldnt be the reposonsibilty of the users.
specifically, websockets are blocked by the "XHR" component on the popup panel, which also blocks XmlRPC and the Fetch API. as a rule, you could block all XHR requests everywhere with:
* * xhr block
What I actually do is block everything but first-party requests, apart from CSS and images:
i actually allow xhr on third-party requests once i enabled third-party requests, which makes it possible to "enable" a bunch of site with two clicks (popup panel then click on "all").
Ublock allows you to block websocket requests. eg.
*$websocket
will block all websocket connections. You probably want to operate on a whitelist on a site by site basis. Blocking localhost or 127.0.0.1 isn't reliable because sites can use dns rebinding attacks to bypass your filters.
Ublock origin says it supports ABP filter rules which allow for whitelisting sites which seems like it should allow something approximating:
~site.com$websocket
*,~site.com$websocket
However this seems like it's invalid syntax because switching your example to this opens all websocket use back up tested via https://websocketstest.com/
Thanks for the information and all that you do. I had found the linked section from UBO to https://help.eyeo.com/en/adblockplus/how-to-write-filters#el... and assumed that *$websocket was <all_domains><separator><rule> similar to how other filters seem to work but I haven't delved too deeply in the rule syntax. Appreciate the correction.
I do this by running my browser in its own network namespace, with NAT to the outside world. A couple more firewall rules explicitly add forwarding for specific localhost ports.
"Whether the port scan is used as part of an infection or part of e-commerce or bank "security checks", it is clearly malicious behavior and may fall on the wrong side of the law."
Though I really don't know what ebay or banks or any site might be doing, it seems like it's almost certainly a defensive thing looking for signs of trouble. I don't know if I'd call it malicious. Isn't this totally harmless in this case? That is, eBay portscans me, how is this malicious?
The greater issue is that browsers are allowing code executing from the public Internet scope (scope meaning security domain) network access to the localhost scope or the Intranet scope (RFC1918 addresses.)
If anything, this should require very explicit permission granting from the user. I’d prefer it be something more like an undocumented toggle accessible solely to developer types.
Yeah, that's the best solution. It should be like microphone or camera access. It should say "this web site is attempting to access a resource on your local system / network."
I don't think you need to overdo it in terms of making the warning red, etc. Just a popup will really discourage people from trying to use this for fingerprinting.
BTW the site says:
"Port scanning is malicious." I don't agree. There are many many things that can look like a port scan but are not malicious, most notably NAT traversal attempts by WebRTC, games, chat apps, and so on.
Right.
"it is clearly malicious behavior and may fall on the wrong side of the law."
It's not against the law. It might be _shady_ but it's not illegal. When I'm teaching cyber intro classes, I let folks know portscanning is NOT illegal but shady. It's like going to a business after hours. It's not illegal to rattle doors and windows to see if they're locked. The police might have a different take on it, but it's not illegal.
It shouldn't matter what malware is on a client device as long as the client has authenticated; the server/company/ebay should be protecting their API from abuse at the API layer, not the client layer.
I think what you’re saying is the user might be an employee on some internal trusted company network. The employer should have control of that browser (and entire endpoint), otherwise the network should likely not be considered trusted. So, in this case, no, the user shouldn’t have the ability to authorize this; the administrator of that browser should.
There are legitimate reasons for port scanning, but I'm not sure most websites out there are using it for noble purposes.
I guess browsers could allow it based on explicit permission from the user, just like it's already done for microphone and camera.
Port scanning from a user’s browser is effectively sneaking behind a user’s firewall. The only legitimate reasons I can envision are security research, and this, to me, is such a small edge case that I’m not sure such access is ever warranted.
I’d be all for a user notification that says “fnord.com wants to access 192.168.0.10 on tcp/443, which seems to be a web server on your home/work network. Are you sure you want to allow this?” I’d want to see this for each new access request, such that port scanning would not be a use case that was supported.
Sure, have an about:config toggle to shut this off, with appropriate warnings.
IRC servers don't run in a browser. Instead they scan ports from the outside, which is not a problem. Anything they find is open to the entire internet anyway.
You run a network, and want to run a security audit. You need to know what devices are operating on it, and what services they are offering.
I don't get upset if someone opens and closes a socket to my VPS to see if something's there. My VPS is exposed to the internet. If a socket opens, it should be secure anyways. There's the chance nginx has an unknown zero day, but if I wanted to avoid that, I'd firewall it.
Things are a little less nice if you open a socket and start sending data to see what's there, assuming the server doesn't respond with a banner.
What about port scanning your service before you've secured it during development? At some point we have to be able to trust the network we're on. It's ludicrous to expect everything to be configured correctly and securely right from the start especially if you're developing the thing being scanned while it's being scanned. I'd much rather websites not be able to scan my home or office network than have to treat that network like I treat the Internet.
But that's a very different use case than having a website you visit portscan your computer (which I believe is what the user above you is referring to.
There's really no legitimate reason for eBay, or any other website, to portscan your computer. There's nothing there needed for browsing their website.
That might be a reason for you to port scan your network.
It is not a reason for your website to port scan my network. Especially since your website running inside my browser is inside my firewall.
> and want to run a security audit
Then you use tools designed to run security audits. You don't open a huge security hole in everybody's browser just so you can use a browser to run a security audit.
> I don't get upset if someone opens and closes a socket to my VPS to see if something's there. My VPS is exposed to the internet.
That's not what's happening here.
My laptop is not exposed to the public internet because it's behind a firewall / NAT. This is like going to my house, plugging a device into an Ethernet port on my router, and scanning my internal network from inside my network.
Except instead of them planting a device, all they have to do is get you to navigate to their webpage. They're getting your laptop to do the port scanning for them, and in so doing, they get access to your internal network. The problem isn't port scanning, the problem is NAT busting.
I'm curious, what would be a good reason to do this? I'm not creative enough to think of anything this enables a site to do that isn't malicious. If I'm running a service on localhost, and that service needs to communicate with the site I'm browsing, surely I could just direct that service to communicate with the site itself.
For instance, if I'm running a local chat application and need it to communicate with the web version, why does the website need to be able to port scan to accomplish this? I can think of other ways to accomplish this that are a lot more secure.
Ubiquiti routers have a fairly magical browser SPA that can run on their domain and talk to local routers. It involves webrtc connections to local addresses.
But I think if same-origin were enforced more strictly, they could have found another way.
It does this most of the time, either through the cloud or direct to the controller. But during setup of the first device on a network it does something direct from the browser to get it connected to the cloud.
Your bot is running a redis server locally, it allows local connections, because it's just a bot, boom.
Taking it a bit further, if we have really smart people involved: the timing of the attempted connections/rejections tell you something about the system that you can use to detect bots/scrapers.
Another example of this being used in the past is to scan for chrome extensions that scrape site content as well. I believe LinkedIn might have gotten hit hard for trying something similar but they were using extension URLs not localhost. Some extensions do spin up localhost services though.
This might be banks trying to detect compromised users. Many "tech support" scams aim to get remote desktop access to users PC's and then have them log in to their bank while scammers are connected. I could see how banks looking for remote access software could be a useful heuristic in fighting this problem.
I'm not defending this use case, but one example I can think of is that Spotify runs a local server so that websites you access can control it, e.g. if you are on Billboard looking at top music charts, clicking on a song could start the song in Spotify, and even embed a player in the web browser, without you needing to be signed into Spotify in your browser.
Yea this is very surprising.
I run a file server in my local network. There is no access control on it because it’s behind my router’s firewall, but everyone in the LAN can access it.
I find it very surprising that now any random website can access it with no oversight.
Why worry about spectre and meltdown when such blatant backdoors are implemented in browsers?!
Does your file server speak HTTP? If so you might have a problem. If not, it sounds like it's inaccessible to this attack, except to discover that it exists.
If the user decides to run a service on their intranet or localhost with a wide open CORS policy, isn't that their choice?
Forgoing CORS and making all inter domain requests user opt-in would make the web experience a lot worse, IMO. Making all intranet or localhost requests user opt-in seems less disruptive.
In the case of scanning, a CORS denial can still reveal information about the user's internal network, as a CORS denial is a different result than a network timeout or a TCP RST.
CORS is set by the target, so localhost CORS policy is directly in the hands of the user. intranet CORS policy is set by whoever operates that intranet service
It does in the sense that you won’t be able to control the websockets payload, so the target server generally won’t respond. The problem here is that the information leak is happening prior to any data being sent over TCP, so the fact that the server will drop the connection as invalid doesn’t help.
WS does have its own way of cross-domain opt-in. I think it users slightly different headers than CORS for historical reasons but effective does the same.
That a script is able to gather information about an origin that did not it in seems like a serious bug to me.
This is because websockets were created after the same origin policy, so they have always sent the origin header that allows the server to filter connections. CORS was only needed because the browsers were adding the same origin policy to HTTP requests that had historically never had it, so they needed some set of rules (and overrides for them) that didn’t break existing websites.
CORS has nothing to help with here. The site doing the scanning is not able to make connections. Rather it's only able to tell if the port is listening or not. CORS would still be listening so they'd get the same info they're getting now.
I stand corrected. I think yours is the correct approach.
How shall origin be defined? I can envision the likes of Microsoft which have many, many second-level domains making calls between them.
We can’t allow the site itself to grant access. How would this be managed, other than “please stop and think what a domain name is supposed to be before spraying your product across twelve of them?”
Granting access down seems ok to me. To make it really generic, you would need a way to query the upper domain if accessing a sibling is ok.
The process is different enough from cookies to warrant another large discussion about how to do it, with plenty of trial and errors. But the stakes are much lower, as in the worst case the user will get a dialog, instead of a site being broken.
To be clear, evil.com can define sub.evil.com to resolve to 127.0.0.1. You basically can’t look at domains to mean anything much. You have to look at IP addresses.
(Which is in turn made harder by IPv6 public addressing, where you can’t just block the private IP range because you might not be behind a NAT in the first place, instead only behind a firewall. So your address A::B, can route to your Intranet peer A::C, which isn’t public-routable, but is a public address. But there’s nothing, other than the firewall, that says that that’s not a public address. It’s a hard problem!)
To add on to your point, even if you allow evil.com to only access evil.com and not any subdomains, your browser is still vulnerable because of short TTLs on DNS resolution.
evil.com can set a short DNS TTL, and after you access it, it can rebind its address to 127.0.0.1. Then subsequent requests to evil.com go to localhost (e.g. fetch("evil.com", ...) on evil.com will go to 127.0.0.1 if the DNS rebound successfully).
Caching a website's IP on first use doesn't help, either, because it breaks long sessions on websites that use DNS rebinding for legitimate purposes (load balancing, fallover).
The only real way to fix this is for the local webserver to check the Host header on the HTTP request... or look at IP addresses. But building a global registry of IP addresses is hard, so we're stuck with trusting application developers (and malware writers) who run servers on localhost to use good security practices.
Evil.com could be set to resolve normally but redirect you to <plausible random word>.evil.com, which resolves normally once and then performs the attack, leaving evil.com able to keep serving new visitors.
We already have a notion of origin that is used for most of the browser security policies (exact match of domain, protocol, port). Websockets allow servers to enforce this policy by sending an Origin header, but unfortunately observing the error messages/timing still allows you to determine if the port is open at the transport layer even if you can’t establish a connection. Since websockets routinely need to connect to different origins (they can’t be routed exactly like normal requests, though many CDNs/reverse proxies can handle both), browsers would need to remove the information leak themselves by normalizing error messages and timing across failures.
Yes, it's very similar to CORS. They just need to block all localhost requests from non-localhost pages. Maybe carve out an exception for when the dev tools are open.
If anyone thinks of implementing this, don't forget to guard against reflection attacks[1]
EDIT: revisiting my comment (and the wikipedia article linked), a reflection or amplification attack in this context is sending traffic and generating (perhaps much more) traffic from a different source than yours as part of an attack. For example, you could spoof the IP address of the HTTP packets and cause the server to port scan another machine -> little traffic (HTTP request) causing a lot of traffic (port scanning). As part of a DDOS attack, a botnet for example could use this to amplify their attack and masquerade the source.
My kids complained today that Google Classroom isn't working. After a quick investigation, I noticed that Snort on my firewall blocked the relevant Google server due to incoming TCP port scans. Sigh.
Be careful with automated rules - unless it's a full TCP handshake, you can't conclusively identify the source of a port scan as the IP may be spoofed. If someone port scanned you and spoofed eg the IPs of your DNS servers, you've self-DoSed yourself.
From the title I assumed this was going to be something else. I remember some sites used to port scan you on registration. This was to check if registrations were from an open proxy, which was a very strong bot indicator. I might be misremembering but I think Slashdot used to do it. There were also some plugins for phpBB forums that did it too. I used one back in the day and it helped quite a bit with spam registrations.
These were external scans - see if the visitor's IP is running an open SOCKS or HTTP proxy for example. Many IRC networks still port scan you on connect for the same reason.
Yes, I definitely remember Slashdot doing it, but it was more often than just on registration. Here's an article from 2014 about it, which says that it happens on every login and posting a comment: https://soylentnews.org/article.pl?sid=14/04/09/1925245
At the bottom of that post it says the code was added on 2008-04-16 19:07:46 +0000.
Every time I hear about some shiny new feature being added to a browser, I think...
1) Will I ever actually use this
2) How is this gonna screw me over
WebSockets, WebBluetooth, WebAssembly, Web-You-Can-Access-my-Accelerometer-and-Battery, haven't ever wanted to use those. Ever. For anything. For any reason. (Edit 3: Oh yeah, I forgot! WebRTC!)
Edit: Fantastic. You can't disable it in Firefox. So what, does Firefox need a freaking iptables implementation now? [1]
"The only theoretical reason for the WebSocket pref these days is the possibility to disable it easily in case there is a security issue found in the protocol itself or so."
The protocol itself is the security issue. ALL OF IT.
Edit 2: So I don't have the time to investigate every new fad when it comes out. I originally thought WebSockets were raw sockets, but they aren't. Firefox blocks access to port 22 -- I was hoping all privileged ports, but it seems just those. Opening a WebSocket to netcat dumps out a HTTP request, so it seems unlikely that you'd be able to talk with anything that doesn't talk HTTP and WebSockets. Firefox also seemingly blocks access to 192.168/24 and 10/8.
This makes me less angry. But what STILL make me angry is that I have to sit and research about some stupid thing that I don't want and can't turn off. Sooner or later, some web dev is gonna argue that all sites should be loaded over WebSockets because his bloated javascript stack performs marginally better, and then WebSockets won't be something I can turn off. Websites will just whitepage.
Edit 4: Done researching this now. I went to ebay on Firefox, and wasn't getting websocket scans. But I've got a stack of uBlock and NoScript... maybe that's interfering with it some how? Opened up a stock config for google-chrome -- that's my browser for "some dumb new web tech that isn't working in Firefox" -- not seeing any scans when I open up inspector and click "WS".
Regardless, his point still stands. You can totally use WebSockets as a port scanner for localhost, assuming the Content Security Policy allows it. Now I gotta go update my nginx configs...
I've had performance issues with umatrix. I tried blocking all JS by default and explicitly enabling scripts. In theory that should make the browser preform better. I suspect there are a bunch of sites that can't run a function or reach a JS resource and then just go into spin loops eating through resources .. either that or the blocking itself is resource intensive.
blocking has non-zero cost, yes. especially if complex whitelists need to be evaluated for every request. but at least for me it still is a net win most of the time.
But you can also operate umatrix in a blacklist-based approach, i.e. simply let most requests through except the categories you deem problematic (e.g. cross-site websockets in this case)
That would be amazing, but it would never happen. Mom and Pop would always click "no" out of fear of the unknown, and my awesome feature wouldn't get used! It has to be enabled by default, they don't know what they're missing!
That’s already how the browser microphone/camera API and notification API work, though. At this point, it’s the expectation/default for new APIs to act this way. The browser makers would just have to be convinced to align the behaviour of these slightly-older APIs to the best-practice UX paradigm for new APIs.
That's literally the opposite of the concern with permission prompts: all the evidence from years of SSL/TLS certificate errors is that users will blindly grant permission.
This is true -- and it may also be the same for miscellaneous permissions. But that doesn't mean it won't be used as an excuse by the feature developers.
Honestly though, this is just me being bitter at web developers.
Right. Firefox is open source. Why is nobody adding prompts for Web Sockets, WebGL, Web Assembly and all the new stuff that a security / privacy concerned user has at least mixed feelings about. Prompts are not a good solution for a wider audience, but at least they satisfy the curiosity of power users and offer the possibility to leave the site if you get the feeling it's too dodgy.
You use a much smaller set of applications than web sites. Moreover, you usually vet your applications and do not run random stuff. Application developers build up trust over time.
Even if I want to use the web as hypertext + some Javascript for interactivity, every stupid web site can pull these shenanigans.
You use a much smaller set of applications than websites because websites exist.
You're imagining a world where the most popular dev environment goes away, and service providers decide to use HTML forms instead of forcing me to download an app every time I want to order a pizza. That world does not exist. The apps aren't going to go away, and your security model can't be, "people just won't install untrustworthy apps."
And put things in perspective here -- we're talking about a security vulnerability that allows port scanning primarily for fingerprinting purposes. A native app can not only port scan, it can literally just make POST requests to those open ports across separate domains. The security risks we're talking about are not even remotely equivocal.
Don't get me wrong, stuff like port-scanning should be fixed in web browsers. But even with these vulnerabilities, the web is still unquestionably the safest consumer-accessible application platform that we have today. Moving applications off of the web and back onto native platforms would be setting security back half a decade.
When someone comes to me and asks how they make their phone more secure and more private, the number one piece of advice I give them, every single time, is "avoid native apps and use websites instead. Don't install Facebook, use the website. Don't install random clicker games, browse them online instead."
The web has been a major asset in my quest to get friends and family not to install a bunch of random malware on their devices. Doubly so when you throw kids and younger users into the equation. I am eternally grateful that the web is advanced enough that people can join a Zoom meeting without installing Zoom on their computer.
Maybe that’s true for you, but if the modern web apps didn’t exist, most average people would be downloading dozens or hundreds of miscellaneous apps. “Oh, I want to order a pizza on my computer, time to download the Domino’s app.” And then now you have Domino’s ads running as daemons on your system.
Server Sent Events and HTTP? With a modern setup it’s going to be sharing an HTTP/2 pipe anyway. Even handles disconnections gracefully/transparently if you’re clever about it.
Can anyone expand on why this technique isn’t more common? I’m so sick of seeing folks reinvent HTTP (poorly) on top of WebSockets. I get if extreme low latency is (allegedly) a requirement.
>Can anyone expand on why this technique isn’t more common?
Hard answer: WS has been around for longer, and it's had more "marketing" for lack of a better word -- more people know about it and know how to use it. Retooling existing WS-using code to use HTTP/2 pipes would be a considerable effort with little to no perceived benefit to most users and teams.
Speculative answer: most web developers live closer to the application and presentation layers, and there's resistance to learning technologies that involve HTTP connection management e.g. in nginx (not to say that there are none of these, there are just fewer of us). WS was at the right place at the right time with a good high-level interface available to the client, and gained traction because of this.
That sounds like the developer's problem, not the browser's problem.
I mean, I could half-ass my work a lot more often if they'd get rid of these burdensome restrictions on cross-origin requests. But they ain't going to.
Domains are never going to be just one origin. If we could force developers (or, more precisely, ops teams) to do this, then CORS would never have needed to exist in the first place, because there would be no need to allow any crossing of origins in the first place.
But, at every point in the web’s evolution (including today), there was always been something that needed to live on a different host or port for some reason or another—usually because it’s too leading-edge for load-balancers to understand how to route it correctly.
TCP load balancers that can handle long-lived flows with connection stickiness et al, are a very modern invention in the web’s history; and even they still stumble when it comes to e.g. WebRTC’s UDP packets, or QUIC.
There is media.peerconnection.enabled in about:config. When set
to false, WebRTC doesn't work but I'm not sure if there isn't
anything left active.
Also uBlock has an option "Prevent WebRTC from leaking IP adresses".
WebRTC should be disabled by default or firefox should ask
explicitly like with webcam-access.
There have already been reports where sites use your browser as a
peer in a P2P-network (without your consent). This can be really
problematic depending on where you live.
Yes, oddly enough. It can be used by a website you visit to gain information about your local network which turns out to be incredibly effective for fingerprinting.
The best solution is running the VPN client in your router. That way, your machine can't see its ISP-assigned public IP address.
If you're using a workspace VM, you can run the VPN client in a pfSense VM, which is functionally equivalent.
If you're not using a workspace VM, and can't run the VPN client in a router, you can run the VPN client in a pfSense VM. You bridge the WAN interface of the pfSense VM to the host LAN adapter. So then the host can't use it. And you configure the LAN interface of the pfSense VM as host-only. So now the host machine (your workspace) can reach the Internet only through the VPN client in the pfSense VM.
Or you can just make sure that WebRTC is disabled.
Websockets are nice for some things. I hack on Mastodon and it uses WSS for streams and they're very helpful.
But WebBluetooth, ASM, etc are all fairly insane. WebRTC feels like a massive security issue (I've seen a demo of someone using WebRTC to find computers on an internal network at a security conference years ago. Even if that hole is fixed, it's still a hacky solution to video streaming behind NAT).
I agree; most of this stuff needs to have ways to disable it, in the base configuration screen of the browser (not hidden somewhere in about:config).
Last I checked (> 1 year ago) it was WONTFIX because of some very idiotic (IMO) reasoning. I keep it permanently disabled and have never missed it (media.peerconnection.enabled in Firefox btw).
No doubt like canvas, font, audio, etc. fingerprinting, it will eventually be required to use Google apps like Maps and Messages for Web (to cite two recent discoveries).
"If you're not paying, you're the product" has its own form of economic inflation.
WASM is great peace of tech but I can't help to think it would be abused a lot in the future. For example, right now we can use ad blocker to block ads and analytics by blocking its js from loading. Imagine when wasm gains mainstream popularity and ad companies begin to ship their ads and analytics product as a libaries to be linked at compile time. How do we block something like that? Sure the adblocker can hide the relevant dom contents, but the code is still run and doing whatever it want on your browser.
These kind of complaints are based on a misunderstanding of how JS works or how the browser works.
You can do the same exact thing in JS right now. In fact, if anything, JavaScript makes this way easier than WASM. With JS, you can just use something like Rollup or Webpack to put your analytics code in the same code.
Yeah but using webpack is not how majority of websites deployed, so it's probably not worth the effort for ads companies to support it. They will consider this when webpack/wasm become mainstream enough (approaching 50% web), which may or may not happen. Probably won't happened but the thought always linger in my mind.
It's easy to first party ad code. I would guess that as a percentage of website usage Webpack, or at least code that uses custom bundling, is over 50%. JS knows all.
Anyone who has enough technical knowledge to have a reason to turn these off should be just fine with about:config. We don't want a situation where normal users just randomly go into the config turning things off because they think they are doing something good. That's how we get another group of low-tech "turn off Windows Update" types who just harm themselves as a result of their incompetence.
> I hack on Mastodon and it uses WSS for streams and they're very helpful.
I'm not familiar with Mastodon or WSS. Can you describe how using WSS make the end user's experience better? What would be different if web sockets weren't used?
Mastodon is a federated/distributed social networking server that communicates with other servers via a protocol called ActivityPub. The interface feels sorta like Twitter, but it doesn't have to be.
There are other FOSS ActiviyPub servers such as Pleroma (written in Elixir), Pixelfed (Instagram type interface) and PeerTube (distributed video). ActivityPub is a protocol (like e-mail/SMTP) for subscribing and replying to posts. ActiviyPub is just used for the backend (how servers communicate with other servers; like SMTP sending e-mail or RSS readers polling RSS).
Mastodon uses websockets to stream posts to the client/web browser. People who make mobile apps and desktop clients also tend to use WSS. It does fall back to regular HTTP polling in case WSS fails. Pleroma and others also re-implement the mastodon API. It means you can have different front-end web clients (Pleroma-fe, soapbox-fe, etc) on top of different backends (Mastodon/Pleroma).
The advantage of WSS is being able to stream new statuses with a socket, rather than polling constantly for new updates.
If you get on a Mastodon/Pleroma/Pixelfed instance (there are hundreds out there or you can setup your own), you can follow me at @djsumdog@hitchhiker.social
WebAssembly is awesome! It's a substantial performance boost, and will allow the off-loading of standards to open-source communities so browser developers can focus on core browser features rather than having to stretch themselves.
The subset of web technologies that seems reasonable to any given speaker often closely matches the subset of web technologies the speaker uses (either for their own code, or in apps they use).
Personally, I'm looking forward to the point that web apps are capable enough to let PWAs do absolutely anything an Android or iOS application could do.
As a side note, noscript has been an eye-opener for how many things are loaded when I open anywebsite! It's been fast, but I've been getting annoyed by guessing what to enable to move past a blank webpage.
And WTF is newrelic and why does half of my internet browsing use it?
Newrelic is an application performance monitoring platform, basically give you performance insight on your webapp, so it gain popularity really fast. Really great for identifying performance bottleneck on the server side. Later on, they launched browser monitoring too, so now all webserver that uses newrelic automatically insert newrelic js analytics on every html endpoint.
With some polish and maintenance it could serve as base for a simple(r) browser for the masses. It is essentially a small wrapper around WebKit, which is far from simple, but it has the benefit of great support for modern web features.
> Opening a WebSocket to netcat dumps out a HTTP request, so it seems unlikely that you'd be able to talk with anything that doesn't talk HTTP and WebSockets.
AFAIK this is only partly true, if the web server does not support the websocket protocol, you cannot connect to it [0].
So if I am understanding this correctly, WebSockets only support a small subset of HTTP and it should therefore not be possible to use them to connect to "classic" HTTP servers or to send GET or POST requests to it.
But you still have to make sure all programs bound to local ports handle WebSocket requests gracefully.
Binding to 127.0.0.1 is a nicely cross-platform way to do inter-process communication (I've done so in the past to mitigate JVM startup/warmup issues).
I've never written this code defensively, because if you run programs that throw random shit at locally-bound ports, that's your responsibility. The web community has decided it's a good idea to give arbitrary websites that capability. It's true that the 'random shit' may only take the form of WebSocket requests, but this is only a minor comfort.
From my perspective, this needs to be locked down.
edit: On second thought, you have always been able to trigger similar requests by e.g. just setting the src attribute of an image: Opera aside, browsers apparently never implemented proper cross-network protections. So from now on, I'll be extra careful to make sure all my servers can handle unexpected, potentially malicious HTTP requests even when bound to 127.0.0.1.
That said, I still do think this is something that needs fixing on the browser-side.
Back in the 90's if you wanted an ohms law calculator you had to go download a poorly written program from some random website. Network admins started locking down what you could download, run, and install due to security problems. Flash became a hit and they started piling on features in the browser so you could run things dynamically without having to download something.
Fast forward almost 30 years and the browser has become so full featured it is practically a weak OS sandbox that allows you to run just about anything. It was originally being extended to avoid that in the first place, and here we are almost back to square one.
yes, because for most user choosing between security and functionality is not a choice they are prepared to reason about.
I'm no expert, but I am computer savvy. My experience is that more security ALWAYS limits functionality that I want. I get around it all with VMs and snapshots. I have totally wide open VMs that have no host access and just reset to the snapshot when I shut them down. the irony is that it is not only harder to choose functionality today, when you want it, but also harder to choose security. The choices are made for you. it is infuriating.
The browser is basically the reinvention of the operating system. Its huge advantage is that it's built on the assumption that the user is trusted and the code isn't. In contrast most operating systems are designed on the assumption that code is absolutely trusted, but the user isn't. That's why rights management in Windows is concerned with who's allowed to access which file, while rights management in Firefox is concerned which which website is allowed to access the Webcam.
The big disadvantage of the browser is of course that there's huge competitive pressure, and most users prefer usability over security. Keeping things secure without asking the user about their intentions every step is a huge challenge (see also Windows UAC, which struggles with the same problem).
> Its huge advantage is that it's built on the assumption that the user is trusted and the code isn't. In contrast most operating systems are designed on the assumption that code is absolutely trusted, but the user isn't.
It's not necessarily an advantage, it's just a different threat model. An OS is protecting against an attacker already having access to the system (whether physically or over network.) The assumption is that the system is working properly and it's the operator that is malicious.
For the browser, the assumption is that the operator is working properly, but the systems they will be accessing are malicious.
The browser security measures are like a guard at the castle gate, allowing or preventing people from entering. The OS security is like locks on the doors inside the castle so that only people with the right keys can get into various protected rooms.
Both are necessary because they're preventing different things (access to the system vs. access once you're already inside the system.)
dont these features allow web apps to compete with native mobile apps? I really don't like the app or play store... so if that is one way to get away from them, that is a good thing.
Whatever happened to a control panel to turn off all of the things you Don't Want? All of these protocols should have triple toggle switches, Enabled, Ask, Disabled. If something on the page doesn't load because that protocol is disabled, it logs to the console so you can turn on what you need for that page.
I get that adding lots of user controls makes state management difficult, but there are tried and true ways to do this; users need the browser to work for them, not the other way around.
Firefox is open-source, so you could add that. I would really appreciate being able to turn WebUSB WebBluetooth and WebRTC off entirely as those expose a lot of devices that I need not to be exposed to the Internet.
Web browsers really are the new operating systems.
Chrome OS was far ahead of its time, both in its general user hostility and in its excessive resource consumption for seemingly mundane tasks. Like when your roommate's YouTube streaming makes your Excel-like app horribly slow, because your new spreadsheet app needs high-speed internet for no obvious reason.
BTW, WebRTC and WebAssembly were great for crypto mining trojans embedded into ads. Before, they had to port their crypto miner to JavaScript by hand. By now, you can just cross-compile it. The progress of technology ... helps only the technology, but not you, the disposable user in front of the screen :p
Which is also funny because, if I remember correctly, IE was deeply integrated into Windows Explorer and the desktop was essentially rendered by the browser. I remember being able to even change my wallpaper to a custom HTML page in Windows
I don't need WebRTC for chat, because IRC is still working just fine.
I don't need WebRTC for video calls, because Skype is still working just fine.
I don't need WebRTC and WebAssembly for online gaming, because I have Steam to install games locally. Plus latency and performance of games emscripten-ed to wasm tends to be atrocious.
So the only uses where I have seen the new Web* APIs in my life was obnoxious ads and newspaper websites trying to shove recommendations and notifications into my face.
What would you want to cross-compile that couldn't be done much better by building a proper local app?
WebRTC enables many of these things on the browser. That is the point. So your "I don't need WebRTC because I use native apps" is basically just "I don't need the internet because I can go next door to talk to my neighbors".
Just because something doesn't have value for you does not suggest it does not have value, period.
Almost. I'd say my argument is "I don't care about an online egg retailer shipping from China, because using the local supermarket is better in every way."
And obviously, Skype and IRC use the internet, too. So what is the benefit to me - the user - of "Skype inside web browser" over "Skype separate of web browser"?
I understand that WebRTC makes "Skype inside web browser" possible, but it also makes a lot of less desirable behavior possible. So there is be a trade-off between the advantages and the risks from adding WebRTC.
Which benefit does WebRTC generate for the user?
I.e. why is it a good thing that web browsers added this?
That's just a matter of browser software. The Web platform should not suffer just because the browser is imperfect. You can write your perfect browser, fairly easily with the libraries available. I will try when I have some time.
Yes, but that is A LOT easier to filter and detect, because then it'll be using the XHR API and HTTP protocol, whereas WebRTC allows free-form protocols.
> WebSockets ... haven’t ever wanted to use those. Ever. For anything. For any reason.
You’ve never used a web-app chat client?
> WebBluetooth
APIs like these don’t exist for the sake of regular unprivileged web-apps. They exist for the sake of browser extensions (or browser “apps”, or apps within a browser-projector like Electron), specifically in order to be used to add driver-like or service-like capabilities to devices like Chromebooks where the browser is the OS.
Before websockets, webapp chat features—as implemented by your average web-backend programmer—couldn’t pass the C10K challenge.
Yes, fundamentally, on a technical level, there’s not much difference between holding open an HTTP connection in long-polling, vs. holding open a websocket connection.
But the abstractions presented by webserver gateway interfaces (e.g. CGI, prefork process-per-connection language-module embeds), languages/web frameworks (e.g. Ruby on Rails), and platforms (e.g. Heroku) back then, just didn’t support long polling efficiently.
HTTP backends, back then, were all built on an assumption of serialized queuing of HTTP requests—with each web server/web stack/worker thread serving requests one-at-a-time, getting each request out of the way as quickly as possible. There was no concept of IO asynchrony. Web servers were request loops, not event reactors. Libuv didn’t exist yet; nginx didn’t exist yet. The standard web server was Apache, and Apache couldn’t “yield” from a request that was idle.
And, as such, providers like Heroku would queue at the load-balancer, and only proxy a single open request to your web backend at a time, under the presumption that it wouldn’t be able to handle concurrent load. So you had to pay for 2x the CPUs (e.g. Heroku “dynos”) if you wanted to be able to hold 2x the connections open!
Entire third-party businesses (e.g. https://pusher.com/) were built around the hosting of custom web servers that were written in an event-driven architecture, and so were able to host these pools of long-polling connections. But they were freakin’ expensive, because even they didn’t scale very well.
Eventually, it was realized that the just-introduced Node.js could do IO asynchrony pretty well, and people started building explicit “websocket servers” in Node, culminating in the https://socket.io/ library. Back then, you couldn’t just put your regular HTTP load-balancer in front of your websocket backend, because your regular HTTP load-balancer almost certainly didn’t support held-open connections. You needed to host socket.io on a separate host/port, directly open to the internet. (This was one of the driving forces of Node’s adoption: as long as you were putting a Node app directly on the Internet, you may as well just put the rest of your HTTP app in there as well, and make the “websocket backend” into your whole backend.)
Sure, these days, every backend, load-balancer, and NAT middlebox can handle long-polling just fine. But we got there with a decade of struggle, and “legacy”-but-not-really code that used WebSockets because they guaranteed the semantics that long-polling couldn’t.
(I should mention, though, that WebSockets still have some advantages in the modern environment; namely, idle WebSockets are known to be idle at the runtime level, and so, unlike with a long-polling HTTP request, a mobile device can relax its wakeup-timer intervals when the only network connections it’s holding open are idle WebSockets.)
> APIs like these don’t exist for the sake of regular unprivileged web-apps. They exist for the sake of browser extensions (or browser “apps”, or apps within a browser-projector like Electron), specifically in order to be used to add driver-like or service-like capabilities to devices like Chromebooks where the browser is the OS.
That's not really true, though: they're part of the Chrome team's belief in not limiting the capabilities of the web as a platform for app development (on the basis of "if you lack one feature the app needs, the entire app ends up native"). This is a large part of Project Fugu: https://developers.google.com/web/updates/capabilities
BOSH is essentially long polling which is pretty difficult to scale (the worst case scenario can become 1 connection per message for a single client).
I'm pretty surprised however, that a nearly 10 year old standard is being considered as "superflous" as newer technology like WebBluetooth and WebUSB. What we had before Websockets wasn't really long polling, it was Flash.
A decade ago people use comet for this, with php and apache! Every single ongoing comet connection occupy an apache process, a significant resource hog, yet people still use it because they have no other choice. These days we have websocket but I bet comet can be implemented with minimal resource penalty these days thanks to proliferation of async webserver support in modern backend stacks.
I didn't mean the "chat with us now" engagements widgets; I meant, like, Google Hangouts, or Slack, or Twitch chat, or even an pre-Google-Docs Etherpad sidebar chat.
Though, honestly, I prefer the web-chat customer service for my bank/cellphone provider/etc. to calling them on the phone. I don't want to wait an hour on hold with my phone using up both battery and minutes; I want to just leave a window open on my computer and have it ding when they're ready.
>> “ Edit 4: Done researching this now. I went to ebay on Firefox, and wasn't getting websocket scans.“
...Just in case you missed it and you’re using Linux, from the article...
“I thought it might be because I run Linux, so I created a new Windows VM and sure enough, I saw the port scan occurring in the browser tools from the ebay home page”
>So I don't have the time to investigate every new fad when it comes out
I'm finding this to be my main problem. I have a fairly solid computer repair/troubleshooting background but can't keep up with the layers of crap that are pasted on top of modern software.
An ongoing list of "features" being added to web browsers and how to get rid of them would be hugely helpful, but I've never found a centralized site for this topic, it's all just scattered around various tech sites.
Actually I think there's middle ground or a way to somewhat satisfy both.
Does Zoom, Slack, or insert-thing-here need the technical ability to portscan localhost in the manner of this blog post? Well, Zoom had their vulnerability making web requests to their app bound to localhost, but did they need to be able to do it? They could get the job done without that functionality.
They already have both an app and a web-only experience. (They try to hide the latter by only presenting the option when the former has evidently failed, but it's still there.) So, it has it both ways by your 1 & 2. Almost as if users should be able to choose.
> it seems many sites are port scanning visitors for dubious reasons.
Claims that in the intro, but then admits ebay is scanning for VPNs, which probably means it's doing fraud detection, which is definitely not a dubious reason, and is probably actually beneficial to the customer.
This is scary. I've always left locally running services unprotected for convenience given they can't be accessed from outside. I can imagine a lot of people running local apps, servers or databases without any auth that could contain sensitive information. Would a webpage be able scrape data from such services? Any way to disable this completely in Firefox and Chrome?
No, webpage javascript is limited to using websocket protocol [1] for connections. That means your database or IP camera, or VoIP phone or router are safe for now. Though the websocket connection establishment seems to allow the javascript to differntiate between a closed and an open TCP socket and a TCP socket that speaks websocket.
Yes, the primary (or only) reason to even implement a WS server connection is exactly to allow data to be scrapped off using a web-browser.
E.g. Asterisk nowadays allows enabling SIP protocol access over websocket so that you can run a javascript VoIP client from inside a browser [1] (and WebRTC for the media layer).
Sure but such servers would always implement authentication when deployed to the web but a lot of such services could run locally unauthenticated to serve local apps. For example services to power electron apps. This tells me data from such services can be stolen very easily.
Electron apps have much more access permissions than a normal website. I don't know technical details of how electron apps interact with the bundled chrome, however this post [1] suggests that electron apps can just talk low-level TCP protocol, no need to wrap everything in an additional websocket protocol layer.
This is bad and should be blocked IMO, at least by default, but can a site do anything other than find out which ports respond to a websocket request? AFAIK they can't send arbitrary network packets. The websocket will only open if the port they are trying to talk to speaks websocket back. This is mentioned in the article.
I'm not saying that's okay. I still don't want them scanning ports on my machine. There might be some services that offer a websocket connection like Plex for example, or the Kinect driver, or Leap Motion. I also don't want them cataloguing ports that are open.
This guy Just has it wrong when he calls port scanning an adversarial technique. It's Just a way to discover Services. You can then use the result to do malicious things but it's not like the only or even main purpose. I humbly refer to this: https://koeln.ccc.de/ablage/portscan-policy.xml (Google translate can help with the german)
To my knowledge, a lot of effort has been put into the design of CORS (and related APIs) to specifically prevent misuse like that. A well-behaved Websocket implementation should not give the calling script any indication why a connection failed.
I know timing oracles are difficult to avoid in many cases - but the technique shown here seems to actually exploit different kinds of exceptions being thrown by the browser.
This seems like a straight-up bug and pretty serious security vulnerability to me.
cors/csp allow the webpage owner to control what servers the javascript running on on their webpage can access and allow web servers to control what 3rd party websites make requests to them.
Notice the missing piece? Neither of those allow the user to control these things.
At the end of the day, it is reasonable to assume that localhost access is a valid security barrier in the general networking sense. Making an exception for certain types of networked applications is just adding a pitfall for some dev to fall into. Good process design has to take into account the inevitability of human error, and leverage things like "forget safe" rather then "remember safe" (forgetting a step should fail safely, with an error or incorrect but still safe behavior, then unsafely, with an exploit or an explosion)
Using websockets or XHR to transverse internet firewalls is browsers transversing security barriers as a feature, and needs to go the way of the mic access, with a per-site prompt.
First of all, fraud detection seems like a legitimate use case here. And WebSockets has many valid uses.
HOWEVER -- how the hell is localhost port scanning allowed to happen without my permission?!
This feels no different from a website trying to check the existence of named directories on my file system or something.
Does WebSockets not require permission to function at all, or shouldn't it be limited to some kind of CORS-type policy or something to connect without a permissions dialog? Or even if it's allowed to port scan the entire public internet, at least block your local machine and network without explicit permission?
I don't follow what this has to do with websockets specifically; they just go over HTTP, so why couldn't you do this with a regular HTTP request?
Either way it seems easy to mitigate at the browser level: block all requests to localhost that don't originate from a page served on localhost. It's not that different from the CORS policy.
447 comments
[ 4.2 ms ] story [ 279 ms ] threadI have to wonder what value eBay would get from port scanning its customers. Is it part of an attempt to detect bots/attackers? Is malware running on their server trying to determine if the client is likely vulnerable to some propagation method?
One time, I port scanned my public IP (of my ISP) from an EC2 box, and I got an email from EC2 saying they received an abuse complaint from the ISP for port scanning activity.
Legitimate usage from researchers and people reading about infrastructure they have the right to do security testing on may be a larger percentage than you think.
What percentage of torrent traffic do you suppose - or better have stats for - is not copyright infringing? I'd think it's about 0%.
Would certainly be interested if you can prove that's wrong.
- Monitoring your own network or that of your clients for exposed ports
- Researching Internet topology, or performing aggregate queries like “how many nginx servers are connected to the Internet”
Can you use it maliciously? Yes. But, most of the time, if you have a target it would make more sense to do the port scan yourself. And if you’re just dragnet searching for vulnerabilities, most you find will probably already have been exploited. Sites like shodan are good for the overall health of the web because they force website owners to maintain security posture. If you know that foregoing a wordpress upgrade means you’re one script kiddy with a shodan account away from getting hacked, you’re going to keep your site up to date. This saves you from script kiddies, but also from the more sophisticated hackers who would run a port scan themselves anyway.
Any legitimate security service is going to be doing there own scans, surely.
Statistics, yes, but I can't see those stats being especially good. You could probably get equally good nGinx data from netcraft, who IIUC get the data from http responses banners on :80 :443.
I'm not sure I buy the "security posture" line, isn't it circular. Tools to help crack your site are good because it means to have to have counter-measures to combat tools for cracking your site?
Only legitimate use of port scanning for me has been testing access to my own/clients computers, I feel. That's not too say I've not used it for illegitimate things ...
1. External network monitoring: know what you have connected to the Internet and get notified if anything changes unexpectedly. This has actually gotten significantly more challenging with services deployed to the cloud where your IT department might not even know which IPs to keep track of.
2. 3rd-party risk assessment: understand the security exposure of your partners, vendors, supply chain or other 3rd-parties. For example, lets say you're an insurance company that wants to provide cyber insurance. Shodan data can help you understand what sort of risk you'd be taking on. The data has also been used in M&A as part of due diligence to get a metric on the security of the IT department of the company they're thinking of acquiring.
3. Market intelligence: basically Netcraft on steroids. Shodan doesn't just have web information but also for many other protocols. This information is used by hedge funds and vendors to understand which products are purchased and deployed to the Internet. The data is skewed due to the nature of public IPs but there are still things you can do.
4. Policy impact: get a measure for how policies at the country-level are impacting Internet connectivity. For example, the OECD used Shodan to get a measure of Internet-connectivity per capita.
5. Fraud detection: is your customer trying to make a purchase from a machine that's been compromised? Or running a a VPN/ proxy? Shodan is used in transactional fraud detection to flag suspicious payments.
Shodan is outside your network's firewall, therefore only able to access services you've exposed to the wider web.
If I understand the article, the websocket scan eBay is doing is trying to connect to local listeners on your laptop, behind your network's firewall and possibly even behind your laptop's firewall.
It does not, however, sound like an attacker can establish arbitrary TCP connections (at least using the technique from the article). Instead, the attacker can determine if something is listening on a port because it will take a different amount of time to negotiate/drop a connection to a port when there is a listener than when there is not a listener.
In other words, this sounds like a variant of a timing attack. As such, presumably, this particular avenue of attack can be mitigated by the browser vendor inserting a delay s.t. no information can be gleaned from how long it takes to negotiate/drop a websocket connection.
EDIT: I also wonder if it would be possible to do a similar port scan using the timing of XHR requests to localhost (e.g. http://localhost:[port]).
Maybe not, but what if the ports you have open actually are HTTP servers for development purposes? In that case wouldn't a website be able to crawl your unreleased work, and/or mess with what you're doing, with requests seemingly "out of nowhere"?
That one you linked is a messed up case. There is a phenomenal podcast that interviews those guys and walks through their engagement. https://darknetdiaries.com/episode/59/
https://www.securityfocus.com/news/126
https://nmap.org/book/legal-issues.html
Edit: Link to the company http://www.forensicstrategy.com/ He also has a data recovery company now http://www.myharddrivedied.com/
LMAO a downvote in 3 seconds. Good old hacker news. Well, downvotes don't make you right. I'm trying to open your eyes and you downvote me. I guess that is to be expected.
https://en.wikipedia.org/wiki/Everything_which_is_not_forbid...
(No, seriously, I know people who believe that.)
I'll be downvoted for saying this.
Because it's against the site guidelines.
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.[1]
[1] https://news.ycombinator.com/newsguidelines.html
I'd say 90% of the time the powers that be at the company had no idea someone was running that software, or that it was still running at their company, and then someone moved a firewall and the system was exposed to more than intended. Then they'd turn it of ... and find another similar tool running somewhere else.
It could be a simple as a test or security system run amok.
This demonstrates the absurdity of the CFAA more than anything else. Sorry for sounding like a broken record but the CFAA is not salvageable and MUST be repealed.
From the article:
> Looking at the list of ports they are scanning, they are looking for VNC services being run on the host, which is the same thing that was reported for bank sites.
> VNC is sometimes run as part of bot nets or viruses as a way to remotely log into a users computer. There are several malware services that leverage VNC for these purposes.
Criminal law is usually not this simple, as most criminal laws will take into account the mental state of the person performing the action.
As for the CFAA that's seemingly down to how aggressive the prosecutor is feeling about your case. I don't think it should be there's no real access happening and unless it's extremely aggressive and degrades network connectivity it's hard to argue there's any real damage done.
That’s what these sites are doing.
You can get to the same answer axiomatically from the text and case history of CFAA (a port scan literally can't grant you the access a CFAA claim needs to prove you intended), but that's obviously treacherous for non-experts to do; instead, the empirical demonstration should be conclusive here.
I don't know why this scan is occurring, but fingerprinting is the most obvious guess, and intrusive fingerprinting performed by real companies is usually about ATO prevention, which means they're not going to tell you any more about it (ATO defense is an arms race).
I think behind the scenes they keep log of some sort of fraud risk, e.g. geoip different from billing country, suddenly a new operating system, vnc/teamviewer running would probably flag your account (even for benign purposes, e.g. you can get your money back or purchase cancelled if that info can prove your transaction was actually unauthorized).
I worked on a ecommerce where the previous developers implemented a rudimentary "score" system like that so that suspicious orders would be put in queue for phone verification (this was pre gdpr)
Trying to open a connection on the other hand it's like trying to open the door. That should be considered as a violation.
That sounds pretty suspicious to me. At least you need the consent of the house’s owner.
You might be doing a legitimate survey (as part of a locksmith business, say)?
I'd strengthen pfundstein's claim; port scanning intrinsically works by trying to open connections. That isn't enough "like" any particular physical thing to make it a correct analogy; it isn't knocking, or walking in, or opening, or anything else. But one thing we can say, without using analogies, is that it is definitely an active effort, an action deliberately taken, not something passive like "looking" is in the real world. That is not on its own proof that it is wrong... I am merely saying, it is certainly active, not passive.
I'm referring to the literal, probably-hundreds-of-person-hours actions taken to create this functionality. This is relevant to both ethical and legal analyses. No analogy.
I think it comes under the Criminal Attempts Act 1981. But it might come down to whether you gather information, if you test a port, intending later - if successful - to attempt an exploit, then that seems like it could be unlawful (under the Law of England & Wales). Keeping a record of ports found with services could be sufficient to demonstrate intent.
[Just to be clear, I'm not endorsing/condemning the law here.]
What are the intents of people doing scan? I don't see any legitimate reason for scanning random people computers or servers that are not yours.
This is kinda gross practice overall.
"Whether the port scan is used as part of an infection or part of e-commerce or bank "security checks", it is clearly malicious behavior and may fall on the wrong side of the law."
Though I really don't know what ebay or banks or any site might be doing, it seems like it's almost certainly a defensive thing looking for signs of trouble. I don't know if I'd call it malicious. Isn't this totally harmless in this case? That is, eBay portscans me, how is this malicious?
If anything, this should require very explicit permission granting from the user. I’d prefer it be something more like an undocumented toggle accessible solely to developer types.
I don't think you need to overdo it in terms of making the warning red, etc. Just a popup will really discourage people from trying to use this for fingerprinting.
BTW the site says:
"Port scanning is malicious." I don't agree. There are many many things that can look like a port scan but are not malicious, most notably NAT traversal attempts by WebRTC, games, chat apps, and so on.
It's not against the law. It might be _shady_ but it's not illegal. When I'm teaching cyber intro classes, I let folks know portscanning is NOT illegal but shady. It's like going to a business after hours. It's not illegal to rattle doors and windows to see if they're locked. The police might have a different take on it, but it's not illegal.
While it can be used to get information to bad things, it itself can be used for good things too.
It shouldn't matter what malware is on a client device as long as the client has authenticated; the server/company/ebay should be protecting their API from abuse at the API layer, not the client layer.
Know your network.
I’d be all for a user notification that says “fnord.com wants to access 192.168.0.10 on tcp/443, which seems to be a web server on your home/work network. Are you sure you want to allow this?” I’d want to see this for each new access request, such that port scanning would not be a use case that was supported.
Sure, have an about:config toggle to shut this off, with appropriate warnings.
Such as?
I don't get upset if someone opens and closes a socket to my VPS to see if something's there. My VPS is exposed to the internet. If a socket opens, it should be secure anyways. There's the chance nginx has an unknown zero day, but if I wanted to avoid that, I'd firewall it.
Things are a little less nice if you open a socket and start sending data to see what's there, assuming the server doesn't respond with a banner.
There's really no legitimate reason for eBay, or any other website, to portscan your computer. There's nothing there needed for browsing their website.
That might be a reason for you to port scan your network.
It is not a reason for your website to port scan my network. Especially since your website running inside my browser is inside my firewall.
> and want to run a security audit
Then you use tools designed to run security audits. You don't open a huge security hole in everybody's browser just so you can use a browser to run a security audit.
That's not what's happening here.
My laptop is not exposed to the public internet because it's behind a firewall / NAT. This is like going to my house, plugging a device into an Ethernet port on my router, and scanning my internal network from inside my network.
Except instead of them planting a device, all they have to do is get you to navigate to their webpage. They're getting your laptop to do the port scanning for them, and in so doing, they get access to your internal network. The problem isn't port scanning, the problem is NAT busting.
A port scanner running on a webpage without the users knowlege is never legitimate.
So the question is what legitimate reason is there for a port scanner running in a web browser with the users knowledge?
For instance, if I'm running a local chat application and need it to communicate with the web version, why does the website need to be able to port scan to accomplish this? I can think of other ways to accomplish this that are a lot more secure.
But I think if same-origin were enforced more strictly, they could have found another way.
Your bot is running a redis server locally, it allows local connections, because it's just a bot, boom.
Taking it a bit further, if we have really smart people involved: the timing of the attempted connections/rejections tell you something about the system that you can use to detect bots/scrapers.
Another example of this being used in the past is to scan for chrome extensions that scrape site content as well. I believe LinkedIn might have gotten hit hard for trying something similar but they were using extension URLs not localhost. Some extensions do spin up localhost services though.
Here's an interesting tangential article about how they get around obstacles with SSL certs for localhost: https://letsencrypt.org/docs/certificates-for-localhost/
I find it very surprising that now any random website can access it with no oversight. Why worry about spectre and meltdown when such blatant backdoors are implemented in browsers?!
That's too little. All access from a different origin should be blocked by default, not only to local nets.
Forgoing CORS and making all inter domain requests user opt-in would make the web experience a lot worse, IMO. Making all intranet or localhost requests user opt-in seems less disruptive.
In the case of scanning, a CORS denial can still reveal information about the user's internal network, as a CORS denial is a different result than a network timeout or a TCP RST.
Still, doesn’t mitigate attacks against non-HTTP speakers.
That a script is able to gather information about an origin that did not it in seems like a serious bug to me.
How shall origin be defined? I can envision the likes of Microsoft which have many, many second-level domains making calls between them.
We can’t allow the site itself to grant access. How would this be managed, other than “please stop and think what a domain name is supposed to be before spraying your product across twelve of them?”
The process is different enough from cookies to warrant another large discussion about how to do it, with plenty of trial and errors. But the stakes are much lower, as in the worst case the user will get a dialog, instead of a site being broken.
(Which is in turn made harder by IPv6 public addressing, where you can’t just block the private IP range because you might not be behind a NAT in the first place, instead only behind a firewall. So your address A::B, can route to your Intranet peer A::C, which isn’t public-routable, but is a public address. But there’s nothing, other than the firewall, that says that that’s not a public address. It’s a hard problem!)
evil.com can set a short DNS TTL, and after you access it, it can rebind its address to 127.0.0.1. Then subsequent requests to evil.com go to localhost (e.g. fetch("evil.com", ...) on evil.com will go to 127.0.0.1 if the DNS rebound successfully).
Caching a website's IP on first use doesn't help, either, because it breaks long sessions on websites that use DNS rebinding for legitimate purposes (load balancing, fallover).
The only real way to fix this is for the local webserver to check the Host header on the HTTP request... or look at IP addresses. But building a global registry of IP addresses is hard, so we're stuck with trusting application developers (and malware writers) who run servers on localhost to use good security practices.
EDIT: revisiting my comment (and the wikipedia article linked), a reflection or amplification attack in this context is sending traffic and generating (perhaps much more) traffic from a different source than yours as part of an attack. For example, you could spoof the IP address of the HTTP packets and cause the server to port scan another machine -> little traffic (HTTP request) causing a lot of traffic (port scanning). As part of a DDOS attack, a botnet for example could use this to amplify their attack and masquerade the source.
[1] https://en.wikipedia.org/wiki/Denial-of-service_attack#Refle...
"This website is trying to access services on your local PC, do you want to allow?"
Or at least as blockers should have a rule for it.
At the bottom of that post it says the code was added on 2008-04-16 19:07:46 +0000.
1) Will I ever actually use this
2) How is this gonna screw me over
WebSockets, WebBluetooth, WebAssembly, Web-You-Can-Access-my-Accelerometer-and-Battery, haven't ever wanted to use those. Ever. For anything. For any reason. (Edit 3: Oh yeah, I forgot! WebRTC!)
Edit: Fantastic. You can't disable it in Firefox. So what, does Firefox need a freaking iptables implementation now? [1]
1 - https://bugzilla.mozilla.org/show_bug.cgi?id=1091016
"The only theoretical reason for the WebSocket pref these days is the possibility to disable it easily in case there is a security issue found in the protocol itself or so."
The protocol itself is the security issue. ALL OF IT.
Edit 2: So I don't have the time to investigate every new fad when it comes out. I originally thought WebSockets were raw sockets, but they aren't. Firefox blocks access to port 22 -- I was hoping all privileged ports, but it seems just those. Opening a WebSocket to netcat dumps out a HTTP request, so it seems unlikely that you'd be able to talk with anything that doesn't talk HTTP and WebSockets. Firefox also seemingly blocks access to 192.168/24 and 10/8.
This makes me less angry. But what STILL make me angry is that I have to sit and research about some stupid thing that I don't want and can't turn off. Sooner or later, some web dev is gonna argue that all sites should be loaded over WebSockets because his bloated javascript stack performs marginally better, and then WebSockets won't be something I can turn off. Websites will just whitepage.
Edit 4: Done researching this now. I went to ebay on Firefox, and wasn't getting websocket scans. But I've got a stack of uBlock and NoScript... maybe that's interfering with it some how? Opened up a stock config for google-chrome -- that's my browser for "some dumb new web tech that isn't working in Firefox" -- not seeing any scans when I open up inspector and click "WS".
Regardless, his point still stands. You can totally use WebSockets as a port scanner for localhost, assuming the Content Security Policy allows it. Now I gotta go update my nginx configs...
umatrix is the layer7 firewall you're looking for, it can block websocket connections, cross-domain ones in particular are quite easy.
But you can also operate umatrix in a blacklist-based approach, i.e. simply let most requests through except the categories you deem problematic (e.g. cross-site websockets in this case)
Google Maps.
Honestly though, this is just me being bitter at web developers.
For applications that want special access to my machine, there SHOULD be a barrier to entry or inconvenience like a client-side installation.
Not everything needs to run in a browser.
This wouldn’t be a meaningful security improvement for anyone.
Even if I want to use the web as hypertext + some Javascript for interactivity, every stupid web site can pull these shenanigans.
You're imagining a world where the most popular dev environment goes away, and service providers decide to use HTML forms instead of forcing me to download an app every time I want to order a pizza. That world does not exist. The apps aren't going to go away, and your security model can't be, "people just won't install untrustworthy apps."
And put things in perspective here -- we're talking about a security vulnerability that allows port scanning primarily for fingerprinting purposes. A native app can not only port scan, it can literally just make POST requests to those open ports across separate domains. The security risks we're talking about are not even remotely equivocal.
Don't get me wrong, stuff like port-scanning should be fixed in web browsers. But even with these vulnerabilities, the web is still unquestionably the safest consumer-accessible application platform that we have today. Moving applications off of the web and back onto native platforms would be setting security back half a decade.
When someone comes to me and asks how they make their phone more secure and more private, the number one piece of advice I give them, every single time, is "avoid native apps and use websites instead. Don't install Facebook, use the website. Don't install random clicker games, browse them online instead."
The web has been a major asset in my quest to get friends and family not to install a bunch of random malware on their devices. Doubly so when you throw kids and younger users into the equation. I am eternally grateful that the web is advanced enough that people can join a Zoom meeting without installing Zoom on their computer.
Can anyone expand on why this technique isn’t more common? I’m so sick of seeing folks reinvent HTTP (poorly) on top of WebSockets. I get if extreme low latency is (allegedly) a requirement.
https://bugzilla.mozilla.org/show_bug.cgi?id=906896 https://bugs.chromium.org/p/chromium/issues/detail?id=275955
Hard answer: WS has been around for longer, and it's had more "marketing" for lack of a better word -- more people know about it and know how to use it. Retooling existing WS-using code to use HTTP/2 pipes would be a considerable effort with little to no perceived benefit to most users and teams.
Speculative answer: most web developers live closer to the application and presentation layers, and there's resistance to learning technologies that involve HTTP connection management e.g. in nginx (not to say that there are none of these, there are just fewer of us). WS was at the right place at the right time with a good high-level interface available to the client, and gained traction because of this.
Because WebSockets is 10 years old and HTTP/2 is 5 years old, and that's not including support in major frameworks for SSE.
I mean, I could half-ass my work a lot more often if they'd get rid of these burdensome restrictions on cross-origin requests. But they ain't going to.
But, at every point in the web’s evolution (including today), there was always been something that needed to live on a different host or port for some reason or another—usually because it’s too leading-edge for load-balancers to understand how to route it correctly.
TCP load balancers that can handle long-lived flows with connection stickiness et al, are a very modern invention in the web’s history; and even they still stumble when it comes to e.g. WebRTC’s UDP packets, or QUIC.
Also uBlock has an option "Prevent WebRTC from leaking IP adresses".
WebRTC should be disabled by default or firefox should ask explicitly like with webcam-access.
There have already been reports where sites use your browser as a peer in a P2P-network (without your consent). This can be really problematic depending on where you live.
A local IP?
If you're using a workspace VM, you can run the VPN client in a pfSense VM, which is functionally equivalent.
If you're not using a workspace VM, and can't run the VPN client in a router, you can run the VPN client in a pfSense VM. You bridge the WAN interface of the pfSense VM to the host LAN adapter. So then the host can't use it. And you configure the LAN interface of the pfSense VM as host-only. So now the host machine (your workspace) can reach the Internet only through the VPN client in the pfSense VM.
Or you can just make sure that WebRTC is disabled.
https://browserleaks.com/
https://webkay.robinlinus.com/
But WebBluetooth, ASM, etc are all fairly insane. WebRTC feels like a massive security issue (I've seen a demo of someone using WebRTC to find computers on an internal network at a security conference years ago. Even if that hole is fixed, it's still a hacky solution to video streaming behind NAT).
I agree; most of this stuff needs to have ways to disable it, in the base configuration screen of the browser (not hidden somewhere in about:config).
Last I checked (> 1 year ago) it was WONTFIX because of some very idiotic (IMO) reasoning. I keep it permanently disabled and have never missed it (media.peerconnection.enabled in Firefox btw).
"If you're not paying, you're the product" has its own form of economic inflation.
You can do the same exact thing in JS right now. In fact, if anything, JavaScript makes this way easier than WASM. With JS, you can just use something like Rollup or Webpack to put your analytics code in the same code.
Almost every site with some kind of front-end framework like React will use a bundler of some kind.
I'm not familiar with Mastodon or WSS. Can you describe how using WSS make the end user's experience better? What would be different if web sockets weren't used?
There are other FOSS ActiviyPub servers such as Pleroma (written in Elixir), Pixelfed (Instagram type interface) and PeerTube (distributed video). ActivityPub is a protocol (like e-mail/SMTP) for subscribing and replying to posts. ActiviyPub is just used for the backend (how servers communicate with other servers; like SMTP sending e-mail or RSS readers polling RSS).
Mastodon uses websockets to stream posts to the client/web browser. People who make mobile apps and desktop clients also tend to use WSS. It does fall back to regular HTTP polling in case WSS fails. Pleroma and others also re-implement the mastodon API. It means you can have different front-end web clients (Pleroma-fe, soapbox-fe, etc) on top of different backends (Mastodon/Pleroma).
The advantage of WSS is being able to stream new statuses with a socket, rather than polling constantly for new updates.
If you get on a Mastodon/Pleroma/Pixelfed instance (there are hundreds out there or you can setup your own), you can follow me at @djsumdog@hitchhiker.social
https://mastodonia.club
Personally, I'm looking forward to the point that web apps are capable enough to let PWAs do absolutely anything an Android or iOS application could do.
In Firefox:
LocalStorage - about:config Dom.storage.enabled
Canvas - block JS via noscript
DRM - Preferences->Disable DRM content checkbox
WebRTC - about:config media.peerconnection.enabled;
As a side note, noscript has been an eye-opener for how many things are loaded when I open anywebsite! It's been fast, but I've been getting annoyed by guessing what to enable to move past a blank webpage. And WTF is newrelic and why does half of my internet browsing use it?
With some polish and maintenance it could serve as base for a simple(r) browser for the masses. It is essentially a small wrapper around WebKit, which is far from simple, but it has the benefit of great support for modern web features.
AFAIK this is only partly true, if the web server does not support the websocket protocol, you cannot connect to it [0].
So if I am understanding this correctly, WebSockets only support a small subset of HTTP and it should therefore not be possible to use them to connect to "classic" HTTP servers or to send GET or POST requests to it.
[0] https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_...
Binding to 127.0.0.1 is a nicely cross-platform way to do inter-process communication (I've done so in the past to mitigate JVM startup/warmup issues).
I've never written this code defensively, because if you run programs that throw random shit at locally-bound ports, that's your responsibility. The web community has decided it's a good idea to give arbitrary websites that capability. It's true that the 'random shit' may only take the form of WebSocket requests, but this is only a minor comfort.
From my perspective, this needs to be locked down.
edit: On second thought, you have always been able to trigger similar requests by e.g. just setting the src attribute of an image: Opera aside, browsers apparently never implemented proper cross-network protections. So from now on, I'll be extra careful to make sure all my servers can handle unexpected, potentially malicious HTTP requests even when bound to 127.0.0.1.
That said, I still do think this is something that needs fixing on the browser-side.
https://developer.mozilla.org/en-US/docs/Web/API/USB
Back in the 90's if you wanted an ohms law calculator you had to go download a poorly written program from some random website. Network admins started locking down what you could download, run, and install due to security problems. Flash became a hit and they started piling on features in the browser so you could run things dynamically without having to download something.
Fast forward almost 30 years and the browser has become so full featured it is practically a weak OS sandbox that allows you to run just about anything. It was originally being extended to avoid that in the first place, and here we are almost back to square one.
I'm no expert, but I am computer savvy. My experience is that more security ALWAYS limits functionality that I want. I get around it all with VMs and snapshots. I have totally wide open VMs that have no host access and just reset to the snapshot when I shut them down. the irony is that it is not only harder to choose functionality today, when you want it, but also harder to choose security. The choices are made for you. it is infuriating.
The big disadvantage of the browser is of course that there's huge competitive pressure, and most users prefer usability over security. Keeping things secure without asking the user about their intentions every step is a huge challenge (see also Windows UAC, which struggles with the same problem).
It's not necessarily an advantage, it's just a different threat model. An OS is protecting against an attacker already having access to the system (whether physically or over network.) The assumption is that the system is working properly and it's the operator that is malicious.
For the browser, the assumption is that the operator is working properly, but the systems they will be accessing are malicious.
The browser security measures are like a guard at the castle gate, allowing or preventing people from entering. The OS security is like locks on the doors inside the castle so that only people with the right keys can get into various protected rooms.
Both are necessary because they're preventing different things (access to the system vs. access once you're already inside the system.)
Chrome the browser is a stalking horse/test harness for ChromeOS.
I get that adding lots of user controls makes state management difficult, but there are tried and true ways to do this; users need the browser to work for them, not the other way around.
I mean, there's a lot in about:config that the settings panel doesn't support.
Chrome OS was far ahead of its time, both in its general user hostility and in its excessive resource consumption for seemingly mundane tasks. Like when your roommate's YouTube streaming makes your Excel-like app horribly slow, because your new spreadsheet app needs high-speed internet for no obvious reason.
BTW, WebRTC and WebAssembly were great for crypto mining trojans embedded into ads. Before, they had to port their crypto miner to JavaScript by hand. By now, you can just cross-compile it. The progress of technology ... helps only the technology, but not you, the disposable user in front of the screen :p
I don't need WebRTC for chat, because IRC is still working just fine.
I don't need WebRTC for video calls, because Skype is still working just fine.
I don't need WebRTC and WebAssembly for online gaming, because I have Steam to install games locally. Plus latency and performance of games emscripten-ed to wasm tends to be atrocious.
So the only uses where I have seen the new Web* APIs in my life was obnoxious ads and newspaper websites trying to shove recommendations and notifications into my face.
What would you want to cross-compile that couldn't be done much better by building a proper local app?
Just because something doesn't have value for you does not suggest it does not have value, period.
And obviously, Skype and IRC use the internet, too. So what is the benefit to me - the user - of "Skype inside web browser" over "Skype separate of web browser"?
I understand that WebRTC makes "Skype inside web browser" possible, but it also makes a lot of less desirable behavior possible. So there is be a trade-off between the advantages and the risks from adding WebRTC.
Which benefit does WebRTC generate for the user? I.e. why is it a good thing that web browsers added this?
Worms. Todays web browser are like MS office 15 years ago. A disaster waiting to happen.
No, cross-compilers to JS are and were thing too, and were used by cryptominers.
What about 172.16.0.0/12?
The web is now a no-install app delivery platform, it makes sense for these things to exist.
Web browsers are the new cross-platform runtime, better to get used to it.
Chrome, OTOH, will happily open a websocket to these IP ranges. Another good reason not to use Chrome.
Why is “I can use websockets on a private network” a bad thing?
You’ve never used a web-app chat client?
> WebBluetooth
APIs like these don’t exist for the sake of regular unprivileged web-apps. They exist for the sake of browser extensions (or browser “apps”, or apps within a browser-projector like Electron), specifically in order to be used to add driver-like or service-like capabilities to devices like Chromebooks where the browser is the OS.
Yes, fundamentally, on a technical level, there’s not much difference between holding open an HTTP connection in long-polling, vs. holding open a websocket connection.
But the abstractions presented by webserver gateway interfaces (e.g. CGI, prefork process-per-connection language-module embeds), languages/web frameworks (e.g. Ruby on Rails), and platforms (e.g. Heroku) back then, just didn’t support long polling efficiently.
HTTP backends, back then, were all built on an assumption of serialized queuing of HTTP requests—with each web server/web stack/worker thread serving requests one-at-a-time, getting each request out of the way as quickly as possible. There was no concept of IO asynchrony. Web servers were request loops, not event reactors. Libuv didn’t exist yet; nginx didn’t exist yet. The standard web server was Apache, and Apache couldn’t “yield” from a request that was idle.
And, as such, providers like Heroku would queue at the load-balancer, and only proxy a single open request to your web backend at a time, under the presumption that it wouldn’t be able to handle concurrent load. So you had to pay for 2x the CPUs (e.g. Heroku “dynos”) if you wanted to be able to hold 2x the connections open!
Entire third-party businesses (e.g. https://pusher.com/) were built around the hosting of custom web servers that were written in an event-driven architecture, and so were able to host these pools of long-polling connections. But they were freakin’ expensive, because even they didn’t scale very well.
Eventually, it was realized that the just-introduced Node.js could do IO asynchrony pretty well, and people started building explicit “websocket servers” in Node, culminating in the https://socket.io/ library. Back then, you couldn’t just put your regular HTTP load-balancer in front of your websocket backend, because your regular HTTP load-balancer almost certainly didn’t support held-open connections. You needed to host socket.io on a separate host/port, directly open to the internet. (This was one of the driving forces of Node’s adoption: as long as you were putting a Node app directly on the Internet, you may as well just put the rest of your HTTP app in there as well, and make the “websocket backend” into your whole backend.)
Sure, these days, every backend, load-balancer, and NAT middlebox can handle long-polling just fine. But we got there with a decade of struggle, and “legacy”-but-not-really code that used WebSockets because they guaranteed the semantics that long-polling couldn’t.
(I should mention, though, that WebSockets still have some advantages in the modern environment; namely, idle WebSockets are known to be idle at the runtime level, and so, unlike with a long-polling HTTP request, a mobile device can relax its wakeup-timer intervals when the only network connections it’s holding open are idle WebSockets.)
That's not really true, though: they're part of the Chrome team's belief in not limiting the capabilities of the web as a platform for app development (on the basis of "if you lack one feature the app needs, the entire app ends up native"). This is a large part of Project Fugu: https://developers.google.com/web/updates/capabilities
BOSH? Awkward, but it works without websockets.
* https://en.wikipedia.org/wiki/BOSH_(protocol)
I'm pretty surprised however, that a nearly 10 year old standard is being considered as "superflous" as newer technology like WebBluetooth and WebUSB. What we had before Websockets wasn't really long polling, it was Flash.
A decade ago people use comet for this, with php and apache! Every single ongoing comet connection occupy an apache process, a significant resource hog, yet people still use it because they have no other choice. These days we have websocket but I bet comet can be implemented with minimal resource penalty these days thanks to proliferation of async webserver support in modern backend stacks.
Nope. Not once. And I've been using the web since Mosaic.
I see business web sites offering to chat with me all the time. I ignore them. If I want to chat, I'll let you know.
Apple's business-to-Messages thing works so well, I hope it puts the scammy webchat companies out of business.
Though, honestly, I prefer the web-chat customer service for my bank/cellphone provider/etc. to calling them on the phone. I don't want to wait an hour on hold with my phone using up both battery and minutes; I want to just leave a window open on my computer and have it ding when they're ready.
...Just in case you missed it and you’re using Linux, from the article...
“I thought it might be because I run Linux, so I created a new Windows VM and sure enough, I saw the port scan occurring in the browser tools from the ebay home page”
I'm finding this to be my main problem. I have a fairly solid computer repair/troubleshooting background but can't keep up with the layers of crap that are pasted on top of modern software.
An ongoing list of "features" being added to web browsers and how to get rid of them would be hugely helpful, but I've never found a centralized site for this topic, it's all just scattered around various tech sites.
1. Why are web browsers becoming application platforms? Web browsers are document readers, and we should treat them as such.
2. Why does Zoom (or Slack, or insert-thing-here) want me to download a native app? I should be able to do it in my web browser!
I sympathize with both philosophies, but they cannot co-exist.
Parent is asking for granular access control over these very advanced and double-edged features, which is a perfectly valid request.
Having permissions is the norm for native apps on mobile, and it's slowly becoming the norm also on desktop, finally.
Browsers are the new OS, they have to implement permissions too without anything enabled by default.
Does Zoom, Slack, or insert-thing-here need the technical ability to portscan localhost in the manner of this blog post? Well, Zoom had their vulnerability making web requests to their app bound to localhost, but did they need to be able to do it? They could get the job done without that functionality.
They already have both an app and a web-only experience. (They try to hide the latter by only presenting the option when the former has evidently failed, but it's still there.) So, it has it both ways by your 1 & 2. Almost as if users should be able to choose.
> it seems many sites are port scanning visitors for dubious reasons.
Claims that in the intro, but then admits ebay is scanning for VPNs, which probably means it's doing fraud detection, which is definitely not a dubious reason, and is probably actually beneficial to the customer.
[1] https://en.wikipedia.org/wiki/WebSocket
E.g. Asterisk nowadays allows enabling SIP protocol access over websocket so that you can run a javascript VoIP client from inside a browser [1] (and WebRTC for the media layer).
[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+Builtin+...
[1] https://stackoverflow.com/questions/41674063/is-it-possible-...
I'm not saying that's okay. I still don't want them scanning ports on my machine. There might be some services that offer a websocket connection like Plex for example, or the Kinect driver, or Leap Motion. I also don't want them cataloguing ports that are open.
I know timing oracles are difficult to avoid in many cases - but the technique shown here seems to actually exploit different kinds of exceptions being thrown by the browser.
This seems like a straight-up bug and pretty serious security vulnerability to me.
Notice the missing piece? Neither of those allow the user to control these things.
At the end of the day, it is reasonable to assume that localhost access is a valid security barrier in the general networking sense. Making an exception for certain types of networked applications is just adding a pitfall for some dev to fall into. Good process design has to take into account the inevitability of human error, and leverage things like "forget safe" rather then "remember safe" (forgetting a step should fail safely, with an error or incorrect but still safe behavior, then unsafely, with an exploit or an explosion)
Using websockets or XHR to transverse internet firewalls is browsers transversing security barriers as a feature, and needs to go the way of the mic access, with a per-site prompt.
HOWEVER -- how the hell is localhost port scanning allowed to happen without my permission?!
This feels no different from a website trying to check the existence of named directories on my file system or something.
Does WebSockets not require permission to function at all, or shouldn't it be limited to some kind of CORS-type policy or something to connect without a permissions dialog? Or even if it's allowed to port scan the entire public internet, at least block your local machine and network without explicit permission?
Edit: https://defuse.ca/in-browser-port-scanning.htm
There doesn't seem to be a way to access anything locally, just test for open ports. I use SSH tunneling a lot and was having a minor freak out.
Either way it seems easy to mitigate at the browser level: block all requests to localhost that don't originate from a page served on localhost. It's not that different from the CORS policy.