Increased load on your web server, but other than that zero! (You might even be able to mitigate the load with caching). It progressively enhances existing links if JavaScript is enabled.
There are other libraries that do similar things, such as InstantClick: http://instantclick.io/
This is a feature available on most web frameworks today (for example Link's prefetch on Next.js), but still could be very useful for smaller website and other static pages not using such frameworks.
I'd be a little wary of using a script from an unknown person without being able to look at the code - I'd rather see this open source before using. Especially being free and MIT licensed, I don't see why it wouldn't be open.
Minification is free and is done only once. In our case, the script is .9kb compressed (2.9kb uncompressed). When minified, it goes down to .6kb compressed (1.1kb uncompressed). It’s a small improvement, but there’s no reason to ignore it.
This should be completely backend agnostic. I was never a Django person, but you’d just put that script tag in your main template so it loads on every page.
It doesn't have to be. Could start by allowing website authors to opt in via a tag in the <head> or something, then opt out on a per-link basis with an attribute (eg preload=false)
Seems like you'd be punishing users instead of website operators since the cause/effect relationship is so unobvious.
User happens to brush over the logout button while using the site. On their next click, they're logged out. Weird. Guess I'll just log in again. Doesn't happen again for some time, but then it does. Weird, didn't that happen the other week? What's wrong with my browser? Oh cool, switching browsers fixed it. You're having that issue, too? Don't worry, I figured it out. Just switch browsers.
There's nothing stopping GET requests from having side effects.
It's like pointing to a list of best practices and saying "everyone surely follows these."
For example, someone changed their signature to `[img]/logout.php[/img]` on a forum I posted on as a kid and caused chaos. The mods couldn't remove it because, on the page that lets you modify a user's signature, it shows a signature preview. Good times.
I think it was a joke as GET requests are not supposed to change anything, but often they do (probably because many devs don't know about, understand or respect the RESTful concept).
EDIT: For completeness, I have to add, that I am also part of the group of people who have violated that concept. Maybe neither frequently nor recently, but I did it too :-/
It's nothing to do with REST. It's part of the HTTP spec and has always been, that "GET and HEAD methods should never have the significance of taking an action other than retrieval".
Well, if I am not mistaken, REST is just the articulated concept on which HTTP was built. So yes, the HTTP spec (probably) existed before REST became a term itself, but in the end, there is no reason to argue if REST defines it or HTTP.
This phrase "GET link" I keep seeing makes sense, but strikes me as odd. Is that to differentiate from an "a" tag that triggers JS that makes a fetch/xhr with another method? The only non-JS non-GET request I'm aware of is a form action (POST by default, GET if specified) which can hardly be called a link, unless I'm wrong to equate link with "a" tag.
Enabling it by default Internet-wide seems like it could be a bad idea for many reasons. If I go out of my way to enable it on my site, I am taking responsibility for the bandwidth and any side-effects of prefetching a link and understand what I am doing. But if it is simply enabled Internet-wide, isn't that bordering on a DDoS? What about poorly-coded websites/apps where GETS are not idempotent or have side-effects? What about server-side analytics/logs tracking HTML downloads?
Indeed, breaking the web by misusing GET is not a good thing. By extension, keeping the web broken by not exposing this breakage is not a good thing either.
Like mentioned in another comment, if somebody used a GET http link to logout from a webpage, you would end up with a ton of surprised users. People who read articles by highlighting the text with the mouse would also probably hover over all of the links and would end up wasting bandwidth for no reason.
> if somebody used a GET http link to logout from a webpage,
If you violate the standards, your website doesn't work. Who knew?
> People who read articles by highlighting the text with the mouse would also probably hover over all of the links and would end up wasting bandwidth for no reason.
"For no reason" is obviously wrong, making the web snappier is a reason.
Maybe browsers should only prefetch links on bloated websites since their owners clearly don't mind wasting bandwidth.
This is just an ignorant response. The history of the internet is littered with pragmatic solutions to standard vs. non-standard approaches for exactly these reasons. See: <image>, Referer header, the HTML standard as a whole.
By the way, if your standard contradicts a popular methodology, it's probably a bad standard.
It's funny how much more sense this made on old-fashioned dial-up connections. Back then, as far as I remember, there was no data limit as such. The only thing that counted was connection time. Rather than sitting there reading something while generating ticks, you could better download much of the site and disconnect. An old form of rush to idle.
It doesn’t work for outgoing links because the gain wouldn’t be as much as the CSS and scripts of the external site need to be loaded in addition to the HTML (only the HTML can be preloaded).
Also, there’s usually no incentive to improve other’s sites pages load.
Kind of. It’s different than InstantClick in that it uses <link rel="prefetch"> and can thus be embedded in one minute, while InstantClick transforms a site into an SPA and require additional work.
It’s a different product. The initially planned name for it was “InstantClick Lite”.
I've seen this kind of features several times on libs. And sometimes I can't un-think about it while my mouse is hovering on links in normal life. It's .. prefetching or not?
Hmm, I wonder: if the user isn't asked to authorise the action whether it technically breaches the Computer Misuse Act (UK).
If you send a requested page, that's obviously fine. Normal use of websites is expected, but unilaterally instructing your site visitor's browser to download further unrequested content that's not part of a requested resource ...?
Where does their code get inserted? I could imagine it might help if it were added to the page you were linking from, but the site seems to indicate the code gets added to page you want to have load quickly -- the page you're linking to.
Why would you simulate hovering over links if you can just visit those URLs directly? If you want all links anyway, you lose the real optimization here, which is not the preloading itself, but limiting that to the (small) subset that a particular user seems interested in.
I don't know that the presence of a query string is a great indicator of whether the link causes state change (cue the typical GET immutability arguments, etc.).
For example, in Drupal every path (whether or not it causes state change) has 2 forms: "/?q=path/to/page" (when you don't have access to .htaccess or .conf) and "/path/to/page" (when you do, and you enable clean URLs).
Embedding via // without explicit SSL should probably be considered harmful or malicious as there is no reason to make such scripts available without SSL. Even if the end website is not using SSL users can still fetch your script securely.
The communication between the user and example.com downloading the page referring to your script is secured by their SSL if they have it.
Separately to that, the communication between the user and your server when downloading your script is secured by your SSL. This can be secure even if example.com is not, so it should only be secure.
If the first html load isn't on SSL, and someone is able to intercept your traffic, they can change the embedded https url to be a non-https url anyway, so I can't even imagine the attack that is prevented by using https into something loaded over http.
Absolutely correct. But this is the website owner's problem and their consequences for not using SSL. You can't help or prevent this because it's not your server, it's not your fault they enabled insecure communication that can be exploited.
When you forgo SSL on your own server someone can also intercept your script in exactly the same way, they don't need to hack the website embedding your script. Now they are your consequences, your fault there's no SSL, and your problem may be affecting everyone who embedded your script insecurely.
No site should be served over plain HTTP in 2019. Browsers and search engines are actively discouraging/downranking websites that don't use TLS at this point.
By the way, since .page is HSTS-preloaded, you may as well include https:// in the code snippet that includes the library. It'll avoid the http-to-https link rewriting internal redirect from happening when included from a non-secure site. It's a tiny performance improvement, but across millions of page views, it might add up.
For exactly this reason browsers don’t send a Referer header when an HTTP request is made from an HTTPS page. (Nor for any kind of request made from a local file.)
You should NEVER load javascript over https on a page that was served over http.
It gives a false sense of security that doesn't exist.
Because the source page was served over http, the source page can be modified by an attacker, making the script be loaded under ssl makes you think its protected. But its not since an attacker could just modify the script tag to remove the https bit in transit then modify the script thats now being loaded over http.
In sort, forcing the script to use https gives you no gain when the page that includes it is served over http and it tricks you into thinking that javascript asset is secure.
Who is being tricked? I don't suspect the user is checking the script tags to see if they are secure. At the very least, this might stop basic ISP tampering used to inject bandwidth warnings, etc.
Who does this affect?
The attacker can always modify the page however they want anyways and the https for the js source would mean the js itself cannot be tampered with
Who still has pages served over anything hit HTTPS?
Someone who wants to explore our emerging financial liability as we grow increasingly compelled by law to actively protect data in transit (that's SSL), at rest, and in distribution.
Penalties for violations can be huge, as much as 20
million euros or 4% of the company's annual turnover.
No browsers present that to users as a secure context, so nobody is being tricked.
More HTTPS == Better. If you can load this over HTTPS you should, no matter what circumstance. The browsers iconography will handle notifying people when the context is secure or not.
FYI, the entire .page TLD is HSTS-preloaded. So in HSTS-preload compliant browsers, this will be rewritten as an https URL prior to fetching it even if included on an http page. In Chrome, you can use web inspector to see the link being rewritten using a 307 'Internal Redirect' prior to sending the request to instant.page.
Not sure how I feel about this. I often hover over a link to see where it is linking to, see if it has a title, etc. But that's probably not typical of most users. And I don't do it on sites I use often and am familiar with.
There's usually no penalty, though. You hover, and the page preload begins. Unless you're trying to keep your data usage to a minimum, there's no disadvantage to you.
Or prevent law enforcement, or other "overseers" from believing you visited a page.
I can see children getting punkd by drive-by prefetch and reporting to teaching staff that X visited a neo-nazi site or, Y downloaded porn during class, etc..
"Prefetch did it" is probably not going to be apparent to most, and is going to sound like a weaksauce excuse.
On the other hand if you're visiting pages that link to neo-nazi content or pornography just one link away from the page you're currently on, chances are the page you're currently on would violate whatever acceptable use policy you're supposed to be following.
Unless whoever is patrolling this filtering is completely insane just show them that page that you were on and how hovering over the link triggers the filter.
Every instance of web filtering I've been subject to in my life just blocks the bad page and the admins expect people to have a few bad requests just by accident or whatever. You'd have to be constantly hitting the filter for it to actually become a real issue.
I feel this fails a user expectation that simply hovering over a link doesn't inform the server of anything.
It's curious you mention checking where links link to, because I think that's also another user expectation failure. The url that appears in the status bar below (or what was once a status bar) is not necessarily the link's true destination. You can go to any google search results page, hover over the links in the results and compare with the href attributes in the <a> tags. They're different. It looks like you'd be going directly to the page that's on the URL, but you're actually first going to google and google redirects you to the URL you saw.
It used to be that checking the url in the status bar allowed you to make sure the link really would take you to where the text made you think it would take you, but that's no longer the case. It seems one can easily make a link that seems like it would take you to your bank and then take you to a phished page.
> I feel this fails a user expectation that simply hovering over a link doesn't inform the server of anything.
I would bet 99%+ of web users do not have a sufficiently detailed mental model of web pages that this is something they've decided one way or the other.
Google Analytics et al, allow custom events which are used to record mouse overs, clicks, et cetera on a majority of websites. I always just assume everything I do, down to page scrolls and mouse movements, is recorded.
>I feel this fails a user expectation that simply hovering over a link doesn't inform the server of anything.
The validity of that expectation died with the advent of web analytics probably two decades ago now. The wholly general solution is to disable javascript, possibly re-enabling it on websites you trust. Sites that break without js are oftentimes not worth browsing anyway.
I hate frameworks not because it's hard or it forced me to do their convention, it's because it's not universal. This is one example. Prefetching should be frameworkless.
Each time you hover over a link it's doing a GET request bypassing the cache (cache-control: max-age), even if you hove the same multiple times.
Also this will make all your analytics false...
Except that indeed this can improve greatly the user sensation of speed
The analytics should only be triggered is the page is rendered, assuming it's done client side. I believe Google does this for first top 3 results if I'm not mistaken.
This helped me quite a bit, my mental model of how this worked was off. Prefetching only downloads the resources but does not actually execute any js code. So sites doing lots of tag managers or other js loading js likely wouldn't benefit, but standard GA, etc. should be fine.
Then how do you know when they actually go to the page? Do you need client side analytics at that point since the browser already has the page in memory?
You could obviously do it with JavaScript by pinging the server to tell him that log entry X is to be promoted to a true "hit".
For a non-JS solution, I guess an tiny iframe at the bottom of the html page that accesses a special server page with an unique stamp that causes the same "hit". the iframe loading would mean that the rest of the page is mostly loaded before it was closed.
Added bonus: these extra pre-loads on hover will tell you if someone nearly clicked a link. Your web server logs contain some poor-man's eye tracking. Could aid determining if important parts of your pages (warning messages and so forth) gather enough attention.
Seems like an invasion of privacy. I'd be surprised if someone hadn't used this sort of script for this already. Perhaps browsers or ad blockers will block this feature in the future.
There's already other libraries out there like Fullstory, which tracks all the user mouse movements and interactions with the page and allows you to watch a user interact with your site in nearly realtime.
There are full tracking tools that do a lot more than this already - tracking all mouse movements and key presses. They are often used with the knowledge of users when testing UI changes/experiments, I've even heard of more general versions running as a browser plug-in for medical monitoring (watching for changes in coordination of people with degenerative conditions, without it feeling like an active test accidentally biasing results), though I would also be surprised not to find the idea already used more widely (and perhaps more nefariously) without users knowing.
If a GET request is all it takes for your analytics to be messed up, then your analytics are not reliable in this era of bots pinging everything.
You need to be emitting your analytics events from a rendered/executed page. Preferably with javascript, and a fallback <noscript> resource link to a tracking url can work here.
Might be worth adding something like a Pre-Fetch: True header to pre-fetched requests. But then the problem is, if the user pre-fetches and then actually views it, how does your analytics know unless the client then sends another request?
In a lot of legit cases, analytics. Especially versus a UX improvement that is clearly minor. The word "analytics" has bad rep but it's massively important to know if you're building the right product for your users, without a sort of analytics you're shooting in the dark
Many people browse the web from an employer who has rules about what
types of pages may be accessed. For example, a person applying for a
job with my team may include a link to a web page about their
job-related background -- portfolio.html or whatever. HR tells us to
be sure we don't follow links to any other page that may be
more personal in nature, such as a page that reveals the applicant's
marital status (which can't legally be considered in hiring decisions
here). HR doesn't want to deal with complications from cases where we
reject an applicant but there's a log entry showing a visit to, say,
family.html from our company's IP address block. We'd prefer that
prefetching isn't a default.
There's also log analysis to identify the set of web pages visited by
each employee during work hours, and an attempt to programmatically
estimate the amount of non-work-related web browsing. This feeds into
decisions about promotions/termination/etc. Prefetching won't get
anyone automatically fired, but we'd still prefer it isn't a default.
The part about viewing family.html seems kind of understandable. If you assume no bad actors, then it's crazy... But we're all developers here, we know that you have to assume the existence of bad actors, and assume that they are going to target you (which is why we you always validate data client data server-side). I could see how viewing family.html could turn into a real headache for HR/Legal, especially if the law says that you can't discriminate based on family information.
The other part about log analysis seems crazy, though, I agree with you on that.
We're talking about evidence in a discrimination court case that points to an IP address associated with a company that visited /family.htm around the time someone applied for a job they didn't get. Like, that person went through their blog's access.log when they got home, defeatedly looking up IP addresses, and going "aha, jackpot!"? And everyone in the company hovers links to be sure they don't go to /as-a-black-man.htm during the hiring process? And the fear seriously is that prefetching might be what spurs this chain of events?
Just go right for the kill and put your marital status, ethnicity, and sexual preferences right below your name on your resume. That way they're trapped the instant they open it!
I don't think it's made up, because I experienced the same thing in a pretty well-known European research center...
Of course, had I known about these practices in advance, I would have declined the job offer. But I didn't. I ended up quitting a few weeks later anyway.
IT would monitor all connections from all employees and send a report to upper management with summary statistics, on a monthly basis.
I was told this was the case by a fellow worker during my second day there, so I tunneled my traffic through my home server via SSH. When IT asked me why I had zero HTTP requests, I reminded them that monitoring employees traffic was illegal under our current legislation. Doing this in a university-like non-profit research center is hard to justify.
I don't see why it's hard to justify. They are providing facilities for you to perform the work they request, not for your personal benefit.
Invert the scenario: if they told you that you had to do work-based research on your own personal Internet connection, would that be OK? Any overage charges are yours to pay, no compensation.
If you need people to design their websites around your companies HR metrics collection for them to work, then your HR department's metrics are the problem. The easiest way to improve productivity at your company could be to drastically cut HR funding, maybe HR should collect some metrics on how much time HR spends building datasets that aren't accurate.
I don't think it's a matter of HR necessarily. HR has guidelines because a lot of their job has to do with compliance. Big companies absolutely need large HR departments because a lot more regulations apply.
While this is an approach you could take, it's not a very user friendly one. I agree with you that those HR policies sound draconian, but OP isn't necessarily in a position to challenge or change them.
In addition to being less user friendly, having the mindset that users/visitors to your website must live and work in ideal settings means that whatever you create will tend to be fragile and brittle because you don't try to take into account situations that you haven't seen before.
I had seen that kind of firewall before when a sale presented what his firewall is capable of at company I worked with. I'm not sure how I felt back then.
I mean that firewall was used to track every website you browse and other evil stuff.
The second hn item today that made me agree that developers need a union.
I always let my team browse Facebook, if they wanted. One of my top people browsed it the most out of everyone. If you block a page, then they will just use their phone.
If you are going to measure, then measure outputs. Measuring inputs will make you and your team equally unhappy.
Measuring Output! Especially true for which their job require more thinking than actual labour. It doesn't matter how much time he spent on it, as long as it gets done. And more often, you actually need to play a game, relax or something different before coming back and solve the problem in seconds as compare to staring at it for hours and not getting anywhere.
One of my biggest career goals is never to work in a company like that. A company that micromanages me to the point they search my fucking internet history to decide my termination doesn't deserve my technical skills.
Sounds terrible: An automated analysis of an employee's behaviour linked to promotions and even terminations? Should be outright illegal.
The webbrowsing of other people is their private matter. If you think someone is surfing the web too much (or taking too many coffee breaks, or leaning on the shovel for a minute or any other normal activity people do to take little breaks from work), it's on leadership to tell the person to get back to work, or generally, create a work environment where work flows more naturally.
Logs are for investigations in case of crimes etc.
I read so many things here on HN that are illegal in Germany. We have laws and powerful worker representation that prevents dehumanitzing stuff like that but, often, things started in the US find their way over here....
That story about 'personal' pages sounds like an excuse for tracking what you do on your computer. I'd look for a different job if I were you to be honest. It sounds like a toxic environment to work in.
> There's also log analysis to identify the set of web pages visited by each employee during work hours, and an attempt to programmatically estimate the amount of non-work-related web browsing.
If your company is doing that, then do not browse on company time and/or using company equipment at all. Ever. They obviously don't (or for regulatory reasons, can't) trust you, so you should treat them as an adversary for your own good.
Remember: HR exists to protect the company, not the employees.
>This feeds into decisions about promotions/termination/etc.
Where do you work that makes tolerating that level of idiotic behavior worth it? It the job super interesting or the pay above market rate? If not, there are much greener pastures my friend.
Companies that treat their employees like morons eventually push out everyone who is not one.
Just wondering, which continent are you located in?
Does your company tell its' employees about their Orwellian policies upfront when hiring, or is it a public secret?
How do you deal with encrypted traffic, e.g. https?
Some companies simply filter web traffic via corporate white/blacklists, maybe you have some insights why an illusion of freedom has been chosen in your company?
> Many people browse the web from an employer who has rules about what types of pages may be accessed.
I think only very few people browse the web from an employer who has rules about what types of pages can be fetched. As a web developer, if I can make a faster experience for 99% of my population at the cost of potentially annoying the HR department of some tiny fraction of them, I'm going to do it. And I won't feel bad about making it slightly more difficult for my site's visitors' management to effectively surveil their browsing habits -- not my problem!
A problem with the loading instructions is that it reveals, to an unrelated site, every single time any user loads the site that is doing the preloading. That is terrible for privacy. Yes, that's also true for Google Analytics and the way many people load fonts, but it's also unnecessary. I'd copy this into my server site, to better provide privacy for my users. Thankfully, since this is open source software, that is easily done. Bravo!
I looked (quickly) through most of the comments below and couldn't answer these questions:
1) What, if anything, is the downside here?
2) Is (Google) analytics effected by the prefetch? That is, does that get counted as a page visit if the link that triggers this prefetch is not actually clicked?
> How? It would load the HTML just as often and not download it a second time as that would invalidate the usage
I'm guessing you didn't read the linked article? It preloads after 65ms on hover, at which point it estimates a 50% chance that the user will click. Hence "loaded twice as much".
> True for users who hover over links they decide not to visit
I've been testing it for the past 30 minutes or so and found that it doesn't cause the same problems that InstantClick did. (Which was javascript errors that would randomly occur.) I'll limit it to a small subset of users to see if any errors are reported but there is a good chance this could go live for all logged in users. Maybe even all website visitors if all goes well.
Seems to have no impact on any javascript, including ads. Pages do load faster, and I can see the prefetch working.
Just make sure you apply the data-no-instant tag to your logout link, otherwise it'll logout on mouseover.
Idempotency is not the issue, the issue is that a user might hover over the logout link, not click it, then move on to the rest of the site and find they are logged out for no reason.
actually, no. Idempotency means that you can safely do the same operation multiple times with the same effect as doing it once. That's a different issue than the no-side-effects rule which GET is supposed to follow.
Also: sometimes a site is misbehaving (for myself, or maybe for a user we're helping) and it's helpful to directly navigate to /logout just to know everyone is rowing in the same direction.
Using a POST, especially if you're building through a framework that automatically applies CSRF to your forms, forecloses this possibility (unless you maintain a separate secret GET-supporting logout endpoint, I guess).
When I originally started my community site I used GET for logout. However, users started trolling each other by posting links to log people out. It wasn't easy to control, because a user could post a link to a completely different site, which would then redirect to the logout link. So, I switched to POST with CSRF and never had another issue.
I've heard this "get requests shouldn't have side effects" argument before, but I don't think it works. At least, not for me, or I'm doing something wrong.
For example: Let's implement authentication, where a user logs in to your api and receives a session id to send along with every api call for authentication. The session should automatically be invalidated after x hours of inactivity.
How would you track that inactivity time, if you're not allowed to change state on get requests?
That brings me back to the year 2001, when my boss's browser history introduced Alexa to our admin page and they spidered a bunch of [delete] links. cough cough good thing it was only links from the main page to data, and not the actual data. I spent the next few days fixing several of problems that conspired to make that happen...
Depends on your definition of “state.” A GET to a dynamic resource can build that resource (by e.g. scraping some website or something—you can think of this as effectively what a reverse-proxy like Varnish is doing), and then cache that built resource. That cache is “state” that you’re mutating. You might also mutate, say, request metrics tables, or server logs. So it’s fine for a GET to cause things to happen—to change internal state.
The requirement on GETs is that it must result in no changes to the observed representational state transferred to any user: for any pair of GET requests a user might make, there must be no change to the representation transferred by one GET as a side-effect of submitting the other GET first.
If you are building dynamic pages, for example, then you must maintain the illusion that the resource representation “always was” what the GET that built the resource retrieved. A GET to a resource shouldn’t leak, in the transferred representation, any of the internal state mutated by the GET (e.g. access metrics.)
So, by this measure, the old-school “hit counter” images that incremented on every GET were incorrect: the GET causes a side-effect observable upon another GET (of the same resource), such that the ordering of your GETs matters.
But it wouldn’t be wrong to have a hit-counter-image resource at /hits?asof=[timestamp] (where [timestamp] is e.g. provided by client-side JS) that builds a dynamic representation based upon the historical value of a hit counter at quantized time N, and also increments the “current” bucket’s value upon access.
The difference between the two, is that the resource /hits?asof=N would never be retrieved until N, so it’s transferred representation can be defined to have “always been” the current value of the hit counter at time N, and then cached. Ordering of such requests doesn’t matter a bit; each one has a “natural value” for it’s transferred representation, such that out-of-order gets are fine (as long as you’re building the response from historical metrics,
If you're not allowed to change state on GET requests, how do you implement timed session expiration in your api? You can't track user activity, in any way, on get requests, but still have to remember when he was last active.
It's not about idempotency, but about side effects. The standards mention if it will cause side effects use POST. Logging out does cause side effect (you lose your login) and hence should be a POST.
In the old days it might have been acceptable to get away with a GET request but these days thanks to prefetching (like this very topic) it's frowned upon.
> In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered "safe".
(there's an exception listed too, but doesn't apply to logout)
EDIT: I know of someone who made a backup of their wiki by simply doing a crawl - only to find out later that "delete this page" was implemented as links, and that the confirmation dialog only triggered if you had JS enabled. It was fun restoring the system.
I don't know why you think I'm contradicting them. I was just pointing out that there are newer RFCs. They also happen to have a stronger and more complete definition of safe methods.
I guess nobody has mentioned this yet, but it presumably doesn't work on mobile?
I wonder if you could do something similar simply by looking at the viewport and loading all the links within the currently visible part of the page. Might be overkill though and end up wasting the user's data.
360 comments
[ 2.8 ms ] story [ 303 ms ] threadThere are other libraries that do similar things, such as InstantClick: http://instantclick.io/
I'd surmise the author is benevolent, but if this were to be turned into a business, some kind of data play seems like the trivial next step.
I'd be a little wary of using a script from an unknown person without being able to look at the code - I'd rather see this open source before using. Especially being free and MIT licensed, I don't see why it wouldn't be open.
The code is not obfuscated or minified, very easy to read.
Both are under MTU size of TCP packet.
[1] https://github.com/instantpage/instant.page
However, I think if browsers had this, but off by default until seeing tags to enable it along with any exclusions, that would be great.
User happens to brush over the logout button while using the site. On their next click, they're logged out. Weird. Guess I'll just log in again. Doesn't happen again for some time, but then it does. Weird, didn't that happen the other week? What's wrong with my browser? Oh cool, switching browsers fixed it. You're having that issue, too? Don't worry, I figured it out. Just switch browsers.
It's like pointing to a list of best practices and saying "everyone surely follows these."
For example, someone changed their signature to `[img]/logout.php[/img]` on a forum I posted on as a kid and caused chaos. The mods couldn't remove it because, on the page that lets you modify a user's signature, it shows a signature preview. Good times.
EDIT: For completeness, I have to add, that I am also part of the group of people who have violated that concept. Maybe neither frequently nor recently, but I did it too :-/
It's nothing to do with REST. It's part of the HTTP spec and has always been, that "GET and HEAD methods should never have the significance of taking an action other than retrieval".
> It's like pointing to a list of best practices and saying "everyone surely follows these."
It’s not a ‘best practice’ it’s literally the spec for the web.
Browsers take a more practical approach than "well, it's in the spec, they should know better" which is apparently what you're suggesting.
It's the same reason browsers will do their best to render completely ridiculous (much less spec-complaint) HTML.
They're already broken, exposing that is a good thing.
If you violate the standards, your website doesn't work. Who knew?
> People who read articles by highlighting the text with the mouse would also probably hover over all of the links and would end up wasting bandwidth for no reason.
"For no reason" is obviously wrong, making the web snappier is a reason.
Maybe browsers should only prefetch links on bloated websites since their owners clearly don't mind wasting bandwidth.
By the way, if your standard contradicts a popular methodology, it's probably a bad standard.
You can't assume a methodology is good just because it's popular. That's how you get cargo cults.
But if a methodology violates the standards, it's almost certainly bad.
Standalone program on Windows XP (or maybe Windows 98?). It had it's own window where you could see which pages it was loading.
Does anyone know the name?
Does this also work for all the outgoing links as well? I don't want to improve other sites rendering time at the expense of my own.
Very cool regardless.
Also, there’s usually no incentive to improve other’s sites pages load.
Off the top of my head, a good way seems to be to write better sites that don't include 10mb of javascript libraries.
But quite an interesting little thing, especially useful for older websites to bring some life into them. The effect is very noticeable.
It’s a different product. The initially planned name for it was “InstantClick Lite”.
If you send a requested page, that's obviously fine. Normal use of websites is expected, but unilaterally instructing your site visitor's browser to download further unrequested content that's not part of a requested resource ...?
Enumerate through a list of pages on your site and use something like Puppeteer to simulate hovering over links on each page.
For example, in Drupal every path (whether or not it causes state change) has 2 forms: "/?q=path/to/page" (when you don't have access to .htaccess or .conf) and "/path/to/page" (when you do, and you enable clean URLs).
Separately to that, the communication between the user and your server when downloading your script is secured by your SSL. This can be secure even if example.com is not, so it should only be secure.
When you forgo SSL on your own server someone can also intercept your script in exactly the same way, they don't need to hack the website embedding your script. Now they are your consequences, your fault there's no SSL, and your problem may be affecting everyone who embedded your script insecurely.
[1] https://developer.mozilla.org/en-US/docs/Web/Security/Subres...
You should NEVER load javascript over https on a page that was served over http.
It gives a false sense of security that doesn't exist.
Because the source page was served over http, the source page can be modified by an attacker, making the script be loaded under ssl makes you think its protected. But its not since an attacker could just modify the script tag to remove the https bit in transit then modify the script thats now being loaded over http.
In sort, forcing the script to use https gives you no gain when the page that includes it is served over http and it tricks you into thinking that javascript asset is secure.
Does it help much? No. Should you use it? Yes.
More HTTPS == Better. If you can load this over HTTPS you should, no matter what circumstance. The browsers iconography will handle notifying people when the context is secure or not.
I can see children getting punkd by drive-by prefetch and reporting to teaching staff that X visited a neo-nazi site or, Y downloaded porn during class, etc..
"Prefetch did it" is probably not going to be apparent to most, and is going to sound like a weaksauce excuse.
Every instance of web filtering I've been subject to in my life just blocks the bad page and the admins expect people to have a few bad requests just by accident or whatever. You'd have to be constantly hitting the filter for it to actually become a real issue.
It's curious you mention checking where links link to, because I think that's also another user expectation failure. The url that appears in the status bar below (or what was once a status bar) is not necessarily the link's true destination. You can go to any google search results page, hover over the links in the results and compare with the href attributes in the <a> tags. They're different. It looks like you'd be going directly to the page that's on the URL, but you're actually first going to google and google redirects you to the URL you saw.
It used to be that checking the url in the status bar allowed you to make sure the link really would take you to where the text made you think it would take you, but that's no longer the case. It seems one can easily make a link that seems like it would take you to your bank and then take you to a phished page.
I would bet 99%+ of web users do not have a sufficiently detailed mental model of web pages that this is something they've decided one way or the other.
Google Analytics et al, allow custom events which are used to record mouse overs, clicks, et cetera on a majority of websites. I always just assume everything I do, down to page scrolls and mouse movements, is recorded.
The validity of that expectation died with the advent of web analytics probably two decades ago now. The wholly general solution is to disable javascript, possibly re-enabling it on websites you trust. Sites that break without js are oftentimes not worth browsing anyway.
What would be a good way to avoid prefetch requests in your analytics if you only derive analytics from server access logs?
For a non-JS solution, I guess an tiny iframe at the bottom of the html page that accesses a special server page with an unique stamp that causes the same "hit". the iframe loading would mean that the rest of the page is mostly loaded before it was closed.
How do you use this library with JS disabled?
You need to be emitting your analytics events from a rendered/executed page. Preferably with javascript, and a fallback <noscript> resource link to a tracking url can work here.
There's also log analysis to identify the set of web pages visited by each employee during work hours, and an attempt to programmatically estimate the amount of non-work-related web browsing. This feeds into decisions about promotions/termination/etc. Prefetching won't get anyone automatically fired, but we'd still prefer it isn't a default.
I've heard a lot of stories of ridiculous rule-by-HR culture, but that's so extreme it sounds made up.
The other part about log analysis seems crazy, though, I agree with you on that.
That sounds batshit insane.
Of course, had I known about these practices in advance, I would have declined the job offer. But I didn't. I ended up quitting a few weeks later anyway.
IT would monitor all connections from all employees and send a report to upper management with summary statistics, on a monthly basis.
I was told this was the case by a fellow worker during my second day there, so I tunneled my traffic through my home server via SSH. When IT asked me why I had zero HTTP requests, I reminded them that monitoring employees traffic was illegal under our current legislation. Doing this in a university-like non-profit research center is hard to justify.
Couldn't you just say "I just dont use http anymore because this X company data is very valuable to me" ?
Invert the scenario: if they told you that you had to do work-based research on your own personal Internet connection, would that be OK? Any overage charges are yours to pay, no compensation.
In addition to being less user friendly, having the mindset that users/visitors to your website must live and work in ideal settings means that whatever you create will tend to be fragile and brittle because you don't try to take into account situations that you haven't seen before.
I mean that firewall was used to track every website you browse and other evil stuff.
I always let my team browse Facebook, if they wanted. One of my top people browsed it the most out of everyone. If you block a page, then they will just use their phone.
If you are going to measure, then measure outputs. Measuring inputs will make you and your team equally unhappy.
So, many years ago, I started hacking away on something that is very appropriate to your username...
https://linuxontheweb.org/desk.os
The webbrowsing of other people is their private matter. If you think someone is surfing the web too much (or taking too many coffee breaks, or leaning on the shovel for a minute or any other normal activity people do to take little breaks from work), it's on leadership to tell the person to get back to work, or generally, create a work environment where work flows more naturally.
Logs are for investigations in case of crimes etc.
I read so many things here on HN that are illegal in Germany. We have laws and powerful worker representation that prevents dehumanitzing stuff like that but, often, things started in the US find their way over here....
If your company is doing that, then do not browse on company time and/or using company equipment at all. Ever. They obviously don't (or for regulatory reasons, can't) trust you, so you should treat them as an adversary for your own good.
Remember: HR exists to protect the company, not the employees.
Where do you work that makes tolerating that level of idiotic behavior worth it? It the job super interesting or the pay above market rate? If not, there are much greener pastures my friend.
Companies that treat their employees like morons eventually push out everyone who is not one.
Does your company tell its' employees about their Orwellian policies upfront when hiring, or is it a public secret?
How do you deal with encrypted traffic, e.g. https?
Some companies simply filter web traffic via corporate white/blacklists, maybe you have some insights why an illusion of freedom has been chosen in your company?
I think only very few people browse the web from an employer who has rules about what types of pages can be fetched. As a web developer, if I can make a faster experience for 99% of my population at the cost of potentially annoying the HR department of some tiny fraction of them, I'm going to do it. And I won't feel bad about making it slightly more difficult for my site's visitors' management to effectively surveil their browsing habits -- not my problem!
P.S: Using HN at my office. Was looking at career pages of other software companies a while ago.
A problem with the loading instructions is that it reveals, to an unrelated site, every single time any user loads the site that is doing the preloading. That is terrible for privacy. Yes, that's also true for Google Analytics and the way many people load fonts, but it's also unnecessary. I'd copy this into my server site, to better provide privacy for my users. Thankfully, since this is open source software, that is easily done. Bravo!
1) What, if anything, is the downside here?
2) Is (Google) analytics effected by the prefetch? That is, does that get counted as a page visit if the link that triggers this prefetch is not actually clicked?
Tia
Client-side analytics like GA aren’t affected.
How? It would load the HTML just as often and not download it a second time as that would invalidate the usage
> this makes for additional load on your server.
True for users who hover over links they decide not to visit
Or am I misunderstanding something here?
I'm guessing you didn't read the linked article? It preloads after 65ms on hover, at which point it estimates a 50% chance that the user will click. Hence "loaded twice as much".
> True for users who hover over links they decide not to visit
Yes, that's the point.
Seems to have no impact on any javascript, including ads. Pages do load faster, and I can see the prefetch working.
Just make sure you apply the data-no-instant tag to your logout link, otherwise it'll logout on mouseover.
Logout links should never be GETs in the first place - they change states and should be POSTs.
I just checked NewRelic, Twilio, Stripe and GitHub. The first 3 logged out with a GET request and GitHub used a POST.
Using a POST, especially if you're building through a framework that automatically applies CSRF to your forms, forecloses this possibility (unless you maintain a separate secret GET-supporting logout endpoint, I guess).
For example: Let's implement authentication, where a user logs in to your api and receives a session id to send along with every api call for authentication. The session should automatically be invalidated after x hours of inactivity.
How would you track that inactivity time, if you're not allowed to change state on get requests?
A GET request should never, ever change state. No buts.
Just because a bunch of well known sites use GET /logout to logout does not make it correct.
Doing anything else as demonstrated in this and other cases breaks web protocols, the right thing to do is:
GET /logout returns a page with a form button to logout POST /logout logs you out
Yes, I was a total n00b in 2001. But then, so was e-commerce.
The requirement on GETs is that it must result in no changes to the observed representational state transferred to any user: for any pair of GET requests a user might make, there must be no change to the representation transferred by one GET as a side-effect of submitting the other GET first.
If you are building dynamic pages, for example, then you must maintain the illusion that the resource representation “always was” what the GET that built the resource retrieved. A GET to a resource shouldn’t leak, in the transferred representation, any of the internal state mutated by the GET (e.g. access metrics.)
So, by this measure, the old-school “hit counter” images that incremented on every GET were incorrect: the GET causes a side-effect observable upon another GET (of the same resource), such that the ordering of your GETs matters.
But it wouldn’t be wrong to have a hit-counter-image resource at /hits?asof=[timestamp] (where [timestamp] is e.g. provided by client-side JS) that builds a dynamic representation based upon the historical value of a hit counter at quantized time N, and also increments the “current” bucket’s value upon access.
The difference between the two, is that the resource /hits?asof=N would never be retrieved until N, so it’s transferred representation can be defined to have “always been” the current value of the hit counter at time N, and then cached. Ordering of such requests doesn’t matter a bit; each one has a “natural value” for it’s transferred representation, such that out-of-order gets are fine (as long as you’re building the response from historical metrics,
> So, by this measure, the old-school “hit counter” images that incremented on every GET were incorrect
Yes they are incorrect. No Buts.
Two requests hitting that resource at the same exact timestamp would increase the counter once if a cache was in front of it.
The whole reason this is supposed to be the case is in order to enable such functionality as this instant thing.
In the old days it might have been acceptable to get away with a GET request but these days thanks to prefetching (like this very topic) it's frowned upon.
https://stackoverflow.com/questions/3521290/logout-get-or-po...
> In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered "safe".
(there's an exception listed too, but doesn't apply to logout)
EDIT: I know of someone who made a backup of their wiki by simply doing a crawl - only to find out later that "delete this page" was implemented as links, and that the confirmation dialog only triggered if you had JS enabled. It was fun restoring the system.
> "Did you even read the article? It mentions that" can be shortened to "The article mentions that."
I wonder if you could do something similar simply by looking at the viewport and loading all the links within the currently visible part of the page. Might be overkill though and end up wasting the user's data.
> On mobile, a user starts touching their display before releasing it, leaving on average 90 ms to preload the page.