HN Explorer

Show HN: JS library to make your website instant

426 points by dieulot ↗ HN
http://instantclick.io/

It seems the internet architecture won't bring us instant websites anytime soon. So here is a hack.

I released this JS library last month (today is version 2.0 release, fixing v1.0's rough edges), it makes use of HTML5's pushState and preloading to make a website instant.

It works like this: before clicking on a link, you'll hover over it. And there's a delay of 200-300 ms between these two events. InstantClick uses this delay to preload a page, when you click on a link it will display instantly in most case. (You can test on the website.)

That's similar to pjax and Turbolinks, for it uses HTML5's pushState and Ajax. And preloading is thrown into the mix.

In case preloading all the links that a user hover over is too much work for the server, there's also an option to set a delay before preloading kicks in, or to preload on "mousedown". Mousedown is when you press your mouse button (a click is when you release it). This way you still get a slight "magical" speed advantage, and pjax's benefits (notably, no recalculation of scripts/styles on every page change).

For mobile it preloads on touchstart, so you get 400 ms to preload. This is different from FastClick: with InstantClick a user can still double tap to zoom without triggering a link.

As said in the first sentence, the internet architecture can't bring us instant websites today or tomorrow. So I think this — InstantClick — could be a pretty big thing to improve the situation at scale.

The main challenge I see is that existing JS scripts may not work with it out of the box. I don't do much web development nowadays though, so I don't know how big of a problem that is, and I don't know if there isn't any other major barrier in the way to make this mainstream.

That's why I'm awaiting your opinion, smart folks of HN.

180 comments

[ 19.0 ms ] story [ 834 ms ] thread
snitko
While interesting, I think this kind of functionality should be implemented only by browser developers and should be turned off by default. Really, I can wait 1 second until the site loads. What I don't want is some library accessing sites without my permission. I usually place mouse over links to see what URL it points to and I sometimes do not wish to click.
dieulot
It works only with links from the website you’re on. When the domain or protocol is different InstantClick don’t (and can’t) preload it.
hrjet
I don't get the "can't" part. Surely you can load it as an image or iframe?
dieulot
Sure, I meant “preload” in the InstantClick sense: getting it ready to replace the DOM.
grandalf
It's no different than a single page app eagerly fetching data...
nawitus
>What I don't want is some library accessing sites without my permission.

Well, in a way all sites already do that by embedding resources from different urls.

callesgg
Realy Cool, Only real problem with this is if the clicking has side effects like: http://example.com/?action=logout as brought up on page.

And probably a ton of other application bugs as style and script stuff wont load. like they normally would

dieulot
You can blacklist these links (or whitelist all the other links).
mrfusion
Would it be worth blacklisting logout by default?

Maybe blacklist any links with the words logout, delete, etc anywhere inside the a tag or in the attributes?

dieulot
No, my philosophy is to not do anything that could get into one’s way. And that’s a false sense of security.

If one is worried about preloading links that triggers an action, there is a whitelist mode to enable links as you review them.

fudged71
Could this be used on every website if it were a browser extension? If it were, I would expect Logout links to be disabled by default.
dylz
Modifying session state should be a POST, not a GET.
callesgg
It is more than often not. Google mail is one example.

But i do concur.

yogo
In theory yes, but I'd wager that most of the time logouts are just hyperlink (/account/signout, /logout, etc.). Unless you had a good reason to, you kinda have to go out of your way to make it a POST.
ecesena
From a security point of view this is not good. An attacker can embed the logout link wherever (e.g. send a tweet) and logout your users. As said in the parent post, GETs should be idempotent and, in particular, not change any state.
yogo
I totally agree. What I'm outlining is that it is very convenient to implement it as a simple anchor tag, hence that is what you usually see in the wild.
bpicolo
Not the libraries' fault if you do something stupid.
hnha
That's what tokens are for but I agree that POST is the way to go.
adambard
I feel using a library like this would constitute a good reason.
windsurfer
So you're saying that poorly-designed applications wouldn't work properly sometimes.
mherkender
Poorly designed applications like this one? https://accounts.google.com/Logout
jodrellblank
Yes.

What's your point? Google must be incapable of poor design? Everything a big company does is good?

mherkender
I wanted to know what HN consensus was on this sort of thing, because it seems to me this is not something that inexperienced programmers do (like SQL injection).

Also, full disclosure: I work for Google.

windsurfer
Yes. Imagine if everyone put the following code on their sites:

    <iframe src="https://accounts.google.com/Logout" width="0" height="0" ></iframe>
snikch
I believe that would be the point of <meta http-equiv="X-Frame-Options" content="deny">
codygman
That's like trying to duct tape your arm back on after losing a fight with a chainsaw.
mike-cardwell
You could do the same with a POST by just running the following in a hidden iframe on your site:

  <form id="form" method="post" action="https://accounts.google.com/Logout"></form>
  <script>$('#form').submit()</script>
The correct way of dealing with this issue is to rely on CSRF tokens.
hnriot
this isn't poorly designed, there are the web specs and then there's the web itself and if it works then the spec need updating, there's no law that post should be used for session state, that was just someone writing it into an RFC, the fact that GET works just fine and a significant population of web apps use GET that way makes it reality more than any spec does.
codygman
Just because using GET to do destructive things "works" doesn't mean the spec should be changed to allow it. By your logic, we should just get rid of all requests except GET.

Someone sends you a link, you click it, it loads in your web browser telling you that you've sent them $5000. That is why POST, PUT, OPTIONS, and all the other methods exist.

This is most certainly poorly designed. Just because most web apps and web app developers suck doesn't mean that the freaking HTTP Standard should be changed.

Just no... please stop.

nimish
It ought to be a PUT since it's idempotent but still modifies state.

It definitely should not be a GET, since good practice says that GET is idempotent (http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html)

Nobody ever follows this since we all want stuff like www.example.com/articles/latest to respond to GET and also have different content depending on time. The distinction is at the core of the REST abstraction and it's an art.

StavrosK
It clearly says that GET's side effects should be the same every time it's run. It doesn't say data retrieved by GET can never, ever change.
codygman
It shouldn't have side effects and be idempotent though. At least if you are adhering to the RFC.
derefr
Oh, are we talking HATEOAS session support?

    # if you don't have a session token yet
    POST /session-tokens # Accept: application/json
      => 200 {"token": "someuuid"}

    # no cookies set!
    # stuff the session token in localStorage instead

    # now use it
    PUT /sessions/someuuid # with your login details
      => 201

    GET /sessions/someuuid/inbox # Accept: application/json
      => 200 {...}

    GET /sessions/someuuid/inbox # again
      => 304 # wow much cached

    # meanwhile
    GET /sessions/otheruuid/inbox
      => 200 {...} # other user: other memcached prefix!

    # pages that won't change per-user...
    GET /home
      => 304 # can be cached globally
             # (use JS client-side, not edge-side, includes)

    # and finally...
    DELETE /sessions/someuuid
      => 301 /home # log out!
Basically, a session is just another resource that you create and then manipulate using its sub-resources. This makes all sorts of things easier: creating session tokens and attaching sessions to them (by logging in, or as a guest) are separate steps, so you can throttle logins at the just by throttling access to the token generator. No cookies and no ESI means caching (reverse-)proxies can actually cache. (Note that I'm presuming TLS and hygienic CORS here, so "session stealing" isn't a valid complaint. Or rather, without those things, it's just as valid a complaint for a cookie-based approach.)

Oh, but as a postscript, one little HATEOAS bonus tweak:

    POST /session-tokens
      => 200 {"token": "/sessions/someuuid"} # no constructing URIs!
tmzt
Something like this should be expanded into an RFC!

It would be great to not have to start from scratch of every web project (even if you you are using a library as you should you often are coming up with the urls for each action.)

Can you expand this with support for password recovery, oauth-like flows, JWT, etc?

dfabulich
Don't do this in real life. Putting session IDs in URLs is not a good security practice. (Session fixation. Users might try to share links containing session IDs. Session IDs leak out via URL history and referrers; URLs show up in the darndest places.)

Roy Fielding thinks that server-side sessions violate the REST constraint. https://groups.yahoo.com/neo/groups/rest-discuss/conversatio...

Some people (maybe you) think he's wrong; why can't sessions just be a resource? (And this just goes to show that nobody knows what "REST" really means, and that this question doesn't matter.)

But if you're willing to treat sessions as resources, there are good security reasons to use cookies to refer to them, ideally in addition to a separate non-cookie ID parameter, to prevent CSRF attacks.

Cookies have some performance benefits, too: they can save you round trips to the server. If I want to display fresh personalized session-specific content on my /home page, why should I force the client to download a generic /home page, then download my JS, then run an AJAX request, when she could just send a cookie in the initial request for /home and get back an (uncacheable) personalized response?

derefr
> If I want to display fresh personalized session-specific content on my /home page, why should I force the client to download a generic /home page, then download my JS, then run an AJAX request, when she could just send a cookie in the initial request for /home and get back an (uncacheable) personalized response?

Your server shouldn't have to do O(N) units of work to serve a static page to N people. The entire point of GET-idempotency is that you can reduce serving your home page to O(1) units of work for you: you render the page, once, and it gets cached by a CDN, like Cloudflare. This necessitates customization either on the client-side, or not at all. Doing anything other than these is breaking the web[1].

(Besides, you don't necessarily have to do an AJAX request for every previously-would-have-been-customized page. Edge-side-include type stuff (e.g. username+id, viewing preferences) is, literally, what localStorage was created to store. You only have to get that type of stuff once.)

> Session fixation. Users might try to share links containing session IDs. Session IDs leak out via URL history and referrers; URLs show up in the darndest places.

As I said, proper TLS and CORS (where things like adding "noreferrer" to all external links is as much a requirement for proper CORS as not loading external images) generally takes care of this.

But you're right, there is a use for cookies. It's not to hold your session token, though. Instead, it's to hold a client fingerprint token, given to the client the first time they speak to you.

A client fingerprint is anything that authenticates a session token. If you say "give me /sessions/foo/..." and you don't send foo's associated fingerprint, the server 403s you at the load-balancer level.

Session fingerprints are already a common concept: some people use the user's IP address, or their browser UA string, or something, as a fingerprint. These have a pretty horrible UX, though, because these things can change unintentionally (e.g. a mobile connection switching cells.) But users expect that clearing their cookies will log them out of things, so the cookie store is a pretty good place to put these. Then, leaking a URL with the session token does nothing, because it doesn't share session secret (the fingerprint), just the identifier.

How is this different from just putting the session token in the cookie store? Mainly in that the client fingerprint doesn't represent you, it represents your machine. The point of it is to pair your browser to the server. You can log in and out as many times as you like, but it'll all happen under the same "client."

Note that if browsers actually chose to start emitting a unique, persistent, per-site client fingerprint as a header (like the iOS "Vendor/Advertising ID"), this would supplant the client fingerprint token -- but do nothing to replace the session token. They're separate things.

Note also that while the session token is part of the URL (and thus part of determining cacheability), the client fingerprint isn't. This should be obvious, but its implication isn't, necessarily: past your load balancer (which authenticates sessions against client fingerprints) cookies cease to exist; they should not be passed to your backend servers. The client, and your backend, operate on pure resources; the client fingerprint becomes a transparent part of the HTTP protocol, invisible to both sides. Gives a much stronger meaning to "HTTPOnly."

---

[1] And it's not that it's bad to break the web, really; you're not hurting anyone else's site than your own when you do this. It's just that HATEOAS gives you some really great guarantees, and pretty much everyone who throws these guarantees away finds themselves re-building the web on top of itself to try to get these guarantees back.

Matryosh...

dfabulich
TLS doesn't address session fixation. (And certainly neither does CORS or the same-origin policy generally.)

Client fingerprinting (with cookies) does address session fixation. But but but didn't you just get through saying how wonderful your solution is because it doesn't use cookies?

If you're happy to use cookies and link them with server-side sessions, then just do that. (Just don't tell Roy.)

And when you need maximum performance, don't force the client to do multiple round trips.

> You only have to get that type of stuff once.

Most visits to public-facing web pages have a cold cache. Maximizing for warm-cache performance at the expense of multiple round trips for cold-cache performance is probably the wrong thing, but only A/B testing and RUM will tell you for sure.

mrfusion
Love it! Will it only work on html5 sites?
dieulot
It works with every websites.
lifeformed
Is there a demo page? I want to see what it feels like.
dieulot
The website it is.
yogo
Good stuff, but I don't think this website is big enough to demo the benefits, maybe a heavier website is needed. A properly configured web server as well as the right HTTP headers will give you the same speed, perceptually.

Also you might want add some kind of tracking to avoid making repeat requests (probably within a timeframe). For e.g. hovering one link, then changing your mind, then going to another link. The browser will smartly pull it from its cache but this lib is still making the requests.

mooreds
I just added it to my database driven website that is run off shared hosting: http://coloradocsas.info/

It doesn't feel that much faster, but I'd love a way to verify speed changes with data.

jwcrux
I'm watching via the dev console and don't see any requests being sent out on hover. Are you sure it's working?
tripzilch
I figured as much, but please add an additional demo part to your website, something like 10 interlinking Lorem ipsum pages. Be sure to also add a few images on some pages (but not on others).

I had already viewed a couple pages (to read what it is about) before I started looking for a demo, and by then I wasn't sure if it was my browser caching or your script that made the pages load snappy.

Maybe also provide a 10-page Lorem ipsum "before" example, clearly marked that it does not use your library, to demonstrate the difference. I can't really tell if maybe your server is just really fast.

VeejayRampay
Another idea: take into account the movement of the mouse to define a directional cone in the general direction of the movement, which would enable you to preload your pages even before the hover state occurs.
dieulot
Kinda overkill. ;) I may explore this idea later, but I don’t have great hope for it.
ivanca
Most times there are some few links that are very frequently clicked on; for example in Hacker News is usually the "next page" and the "threads". So if there is not one already there should be a way to specify some (obligatory?) preload pages.
dieulot
In my opinion even that is overkill. Remember, in most cases it’s already instant. The return on investment (speed/requests) abruptly declines with brute preloading.
CoryG89
I think you should utilize the prefetch behavior of the link tag for situations such as these. No need for JavaScript at all.
VeejayRampay
It is overkill, you're absolutely right about that. But that's the good thing about the webs, you can throw the stupidest ideas on the table and the let everyone flesh it out :D
agumonkey
Weird I thought about this last night and wondered if browser pruned the dom/layers based on input device deltas.
Vektorweg
Or just load every page. ;)
yogo
That might not be very nice for some mobile users that are on a site with a ton of links (like a news site) :)
hakcermani
There is no hover for mobile. Maybe track eye focus in future :-)
yogo
It's a continuation of a joke with the parent comment regarding loading every page. It would be the brute force version of InstantClick :)
dhoulb
Not a bad shout. Not EVERY page, but if you have a five page site, you might as well pull in all five at the time, and save the other four requests later.

The actual HTML is only going to add another few KB.

dclara
I'm sorry that I don't feel it very efficient, because sites with dynamic contents like HK or eCommerce/financial sites, very page is different from minute to minute.

But caching the skeleton of the page will work. We need browser to do it for us, just like browsers already caches the images.

baddox
Or the pages that are most likely to be requested, based on previous session data which you have stored and analyzed.
devgutt
Amazon have done this to improve the performance of dropdown menu. http://bjk5.com/post/44698559168/breaking-down-amazons-mega-...
yelnatz
This article was really eye opening when I read it a few months ago.

I've always hated menu dropdowns where it would close if you try to skip to the next submenu you've opened, since you hovered on the other parts on your way to the submenu.

So you'd end up having to perfectly 'trace' the menu so it won't close down on you.

joshuahhh
That article says nothing about preloading pages. The speed gains are purely a UI thing -- if you're clever about keeping submenus open even when the mouse leaves the submenu's entry in the main menu, you don't need to provide a delay for menu opening/closing.
dmethvin
I think the Amazon example was being used as an example of the "directional cone", not preloading.
mcpherrinm
A few years ago, I went to a tech demo/hackathon thing. Somebody had some neat experiments with preloading pages speculatively based on user behaviour

The coolest approach given was when a user types in the search box, kick off the request for the full results page if they move their mouse on a motion curve towards one of the results, with some machine learned data on how people move their mouse: Not quite in a straight line, and with some acceleration.

zarex
I believe I read somewhere that Amazon does this for their menus. Clever idea to utilize this kind of analysis though.

edit: @devgutt else already mentioned this. I had this page open for a while before posting.

fiatjaf
Another idea: preload all links of every page.
udfalkso
Very nice. It would be great if the JQuery Mobile folks would integrate this.
[deleted]
(comment deleted)
grobmeier
Good idea. Pre-Loading a page when you hover a link with your finger.
CoryG89
After looking at the source, one thought I have is that since you are dealing with such small timescales you should use the high resolution window.performance.now function (or the Date.now function for higher compatibility) as a timer instead of using the Date object as you do.
dieulot
Thanks, I’ll look into it.
napolux
Wonder what happens for website with zillion of visitors per day. Could all this preload impact on servers?
dieulot
Yes. I wonder too what the additionnal load is, I don’t have numbers for now.
napolux
I work for a gazillion pageviews website. I'm discussing this library with my team I'll let you know. :P
dieulot
Thanks, much appreciated!
aphexairlines
Predictive prefetching (similar to the work here, but more aggressive) did impact the servers on a zillion-visitors mobile commerce site at first, but it's fine now. Tuning helps.

More than scaling, the larger headaches come from migrating all your existing logging to account for preloading versus impression hits.

dclara
Oh, good point. CPC is based on the user clicks, i.e., fetching.
lmartel
This is very cool!

One interesting reaction I had: things loaded so fast that I didn't notice one of the page changes and thought it was stuck. For sites like this one where different pages look very similar, maybe it could be worth experimenting with some sort of brief flashing animation (to make it look like a real page load)?

dieulot
Yes, I was thinking about something in the like of NProgress: http://ricostacruz.com/nprogress/ If the page is displayed instantly, show a complete bar that disappears. (I’m not sure if that would solve the problem, I may explore different solutions.)
oneeyedpigeon
Wouldn't this ruin usage stats?
sunir
Careful. Usage trumps usage stats. If the user experience is better, the stats will adapt.
corin_
It doesn't trump the ability to record stats. When it comes to business there are certain things you can't do without, for most sites (that are big enough to matter) stats are absolutely required, no improvement on usage would be worth losing stats for.

(Not talking about this project in specific as it seems stats can be made to work with it, just replying to the idea in your comment.)

philsnow
I haven't read the source for the linked library yet, but could you just have javascript in your pages that hits a /gen204 page on your site when the DOM is ready ? Then the preloaded page will only hit the /gen204 if it's actually rendered. Then just count the 204s instead of 200s.
kjannis
Does this require server components? Or does it also work with a static site?
dieulot
It just requires the 1.6kb JS file. :) So yes, it works on static websites (instantclick.io is hosted on GitHub Pages).
[deleted]
(comment deleted)
soundoflight
Prefetching really shouldn't be blindly applied to everything as users may have bandwidth limited. Even though your implementation is better than browser prefetch on users it does take the choice away from the user unless individual sites make it easy for users to opt out.
dieulot
I think that if you stop hovering fast enough and the request is cancelled, you’ll end up only sending headers, thus not affecting bandwidth that much. Not sure though.

If not the case, note that it’s just HTML we’re wasting. In the grand scheme of things it seems to me it would not have that much of an impact on bandwidth usage.

dclara
I think it may impact on the bandwidth usage on mobile significantly.
samhoggnz
On mobile you don't have a hover state so surely this would be less of an issue.
[deleted]
(comment deleted)
matysanchez
Any demo? I mean, a implementation in a real web, like a blog or something like that?
dieulot
Unfortunately none that I know of.
augustohp
Extra kudos for not using jQuery!
aehv
Is it possible to make it a Chrome extension and use it on all sites?
dieulot
I think it is possible for a subset of websites (because you can’t preload all links by default, some trigger an action such as a logout), that’s an idea I had in mind.
aehv
I pasted the js code into a userscript.

Works great! I hope I won't have too many accidents with it.

I remember using 10 years ago a similar Windows program that preloaded all links.

[deleted]
(comment deleted)
mbesto
In theory you could write a chrome extension and use it for any site, right?
dieulot
Sort of, see my comment below: https://news.ycombinator.com/item?id=7201725
mbesto
Cool, and it only works on GET responses right? So I don't have to worry about hitting "Delete" or "Save" on a web app by hovering.
dieulot
Right. If the “delete” or “save” button isn’t a link it won’t get preloaded.
pokstad
I have an even better hack. Since most blog posts / articles are nothing more than a bunch of text, I simply download all articles in a single fetch when the initial page loads. I do this using a CouchDB view that returns all blog posts in chronological order. All successive link clicks don't hit my server (unless there's an image in the article that needs to be loaded). Check it out: http://pokstad.com
ngokevin
What will you do once you get more than a handful of articles?
pokstad
My thinking is that it takes a LOT of text to equal one good size pic. When using CouchDB as a backend, you want to optimize your queries so that you retrieve as much as possible in the fewest number of requests. With my current list of articles, I have this much data being requested in a single request to my map-reduce view: http://pokstad.com/db/_design/blog/_view/posts_chrono?includ...

That's about 33KB for 9 blog posts (not including attachments). So about 4KB per post. I could have a hundred articles and it would still be under half a MB. To put things into perspective, the cover photo of my pup is about 818KB. There are also numerous resource files, like JS, HTML, CSS that needs to be downloaded before the site will even be ready.

bikamonki
I agree. I am building something similar for a client but I am using node+mongo+backbone. I fetch the 'entire' site sans images and navigation is instant except I place loading anims where an image will show. Obviously not an architecture that suits all needs but very useful for a wide range of semi-static sites. In a way I feel this architecture is like the new 'flash': let the users wait for a bit while everything loads and then let them navigate very fast. (I also use require so some views may need adittional async loads like html templates or js but they are soooo small that the extra ms don't matter)
pokstad
We're going to see this type of architecture more and more. Everyone keeps talking about this C10K problem, but the truth of the matter is that most consumer's computers are idling 99% of the time. The server only needs to be there as central storage and as an authority to authorize certain transactions.
lintiwen
I have problem understand "instant website",

can you provide some specific definitions? thank you

wiradikusuma
Does it work with SPA, particularly using AngularJS? (Essentially what's needed is the "prefetch on hover")
dieulot
I haven’t tested, but I don’t think so. Creeping in the standard browser behavior is easy enough, but an SPA may likely need to reimplement InstantClick’s logic specifically for it.
pt
Yeah, just tried it. Does not work with angular SPA.