Would there be any technical benefit to this over using server sent events (SSE)?
Both are similar in that they hold the HTTP connection open and have the benefit of being simply HTTP (the big plus here). SSE (at least to me) feels like it's more suitable for some use cases where updates/results could be streamed in.
A fitting use case might be where you're monitoring all job IDs on behalf of a given client. Then you could move the job monitoring loop to the server side and continuously yield results to the client.
Good point! We did consider SSE, but ultimately decided against it due to the way we have to re-implement response payloads (one for application/json and one for text/event-stream).
I've not personally witnessed this, but people on the internets have said that _some_ proxies/LBs have problems with SSE due to the way it does buffering.
> we have to re-implement response payloads (one for application/json and one for text/event-stream)
I am curious about what you mean here. The 'text/event-stream' allows for abitrary event formats, it just provides structure for EventSource to be able to parse.
You should only need one 'text/event-stream' and should be able send the same JSON via normal or SSE response.
One of the drawbacks, as I learned - SSE have limit on number of up to ~6 open connections (browser + domain name). This can quickly become a limiting factor when you open the same web page in multiple tabs.
Syncing state across multiple tabs and windows is always a bit tricky. For SSE, I'd probably reach for the BroadcastChannel API. Open the SSE connection in the first tab and have it broadcast events to any other open tab or window.
I tried using SSE and found it didn't work for my use case, it was broken on mobile. When users switched from the browser to an app and back, the SSE connection was broken and they wouldn't receive state updates. Was easier to do long polling
Not sure about the default `EventSource` object in JavaScript, but the Microsoft polyfill that I use (https://github.com/Azure/fetch-event-source) supports `POST` and there's an option `openWhenHidden` which controls how it reacts to users tabbing away.
The standard way to fix that is to send ping messages every ~15 seconds or something over the SSE stream. If the client doesn’t get a ping in any 20 second window, assume the sse stream is broken somehow and restart it. It’s complex but it works.
The big downside of sse in mobile safari - at least a few years ago - is you got a constant loading spinner on the page. Thats bad UX.
That's the dumbest thing I've ever heard. SSE is just a response type for normal HTTP. Explain exactly what you mean cause that's like saying that are removing response types from HTTP.
Refreshing to be reminded of a relatively simple alternative to websockets. For a short time, I worked at a now-defunct startup which had made the decision for websockets. It was an app that would often be used on holiday so testing was done on hotel and restaurant wifi. Websockets made that difficult.
I feel like WebSockets are already as simple as it gets. It's "just" an HTTP request with an indeterminate body. Just make an HTTP request and don't close the connection. That's a WebSocket.
Connections are dropped all the time, and then your code, on both client and server, need to account for retries (will the reconnection use a cached DNS entry? how will load balancing affect long term connections?), potentially missed events (now you need a delta between pings), DDoS protections (is this the same client connecting from 7 IPs in a row or is this a botnet), and so on.
Regular polling great reduces complexity on some of these points.
Long polling has nearly all the same disadvantages. Disconnections are harder to track, DNS works exactly the same for both techniques, as does load balancing, and DDoS is specifically about different IPs trying to DoS your system, not the same IP creating multiple connections, so irrelevant to this discussion.
Yes, WS is complex. Long polling is not much better.
I can’t help but think that if front end connections are destroying your database, then your code is not structured correctly. You can accept both WS and long polls without touching your DB, having a single dispatcher then send the jobs to the waiting connections.
Well, it’s the same in both cases. You need to handle disconnection and reconnection. You need a way to transmit missed messages, if that’s important to you.
But websockets also guarantee in-order delivery, which is never guaranteed by long polling. And websockets play way better with intermediate proxies - since nothing in the middle will buffer the whole response before delivering it. So you get better latency and better wire efficiency. (No http header per message).
That very in order guarantee is the issue. It can't know exactly where the connection died, which means that the client must inform the last time it received an update, and the server must then crawl back a log to find the pending messages and redispatch them.
At this point, long polling seems to carry more benefits, IMHO. WebSockets seem to be excellent for stable conditions, but not quite what we need for mobile.
> It can't know exactly where the connection died, which means that the client must inform the last time it received an update, and the server must then crawl back a log to find the pending messages and redispatch them.
I don't see how this is meaningfully different for long polling. The client could have received some updates but never ack'd it successfully over a long poll, so either way you need to keep a log and resync on reconnection.
Using websockets with graphql, I feel like a lot of the challenges are then already solved. From the post:
- Observability: WebSockets are more stateful, so you need to implement additional logging and monitoring for persistent connections: solved with graphql if the existing monitoring is already sufficient.
- Authentication: You need to implement a new authentication mechanism for incoming WebSocket connections: solved with graphql.
- Infrastructure: You need to configure your infrastructure to support WebSockets, including load balancers and firewalls: True, firewalls need to be updated.
- Operations: You need to manage WebSocket connections and reconnections, including handling connection timeouts and errors: normally already solved by the graphql library. For errors, it's basically the same though.
- Client Implementation: You need to implement a client-side WebSocket library, including handling reconnections and state management: Just have to use a graphql library that comes with websocket support (I think most of them do) and configure it accordingly.
I hope (never needed this) client implementations that do this all for you and pick the best implementation based on what the client supports? Not sure why the transport is interesting when/if you have freedom to choose.
Yeah there’s plenty of high quality websocket client libraries in all languages now. Support and feature are excellent. And they’ve been supported in all browsers for a decade or something at this point.
I vomit in my mouth a bit whenever people reach for socket.io or junk like that. You don’t want or need the complexity and bugs these libraries bring. They’re obsolete.
Using graphql comes with IRS own list of challenges and issues though. Its a good solution for some situations, but it isn't so universal that you can just switch to it without a problem.
I didn't claim that it would be a universal solution. It was just an observation of mine, especially in the context of the mentioned com[arison of their long polling approach vs ElectricSQL.
yeah, authentication complexity with WebSockets is severely underappreciated. We ran into major RBAC headaches when clients needed to switch between different privilege contexts mid-session. Long polling with standard HTTP auth patterns eliminates this entire class of problems.
Yeah, and you can send cookies in the websocket connection headers. This used to be a problem in some browsers iirc - they wouldn’t send cookies properly over websocket connection requests.
As a workaround in one project I wrote JavaScript code which manually sent cookies in the first websocket message from the client as soon as a connection opened. But I think this problem is now solved in all browsers.
I like long polling, it’s easy to understand from start to finish and from client perspective it just works like a very slow connection. You have to keep track of retries and client-side cancelled connections to have one but only one (and the right one) of requests at hand to answer to.
One thing that seems clumsy in the code example is the loop that queries the data again and again. Would be nicer if the data update could also resolve the promise of the response directly.
You could have your job status update push an update into an in-memory or distributed cache and check that in your long poll rather than a DB lookup, but that may require adding a bunch of complexity to wire the completion of the task to updating said cache. If your database is tuned well and you don’t have any other restrictions (e.g. serverless where you pay by the IO), it may be good enough and come out in the wash.
Hard disagree. Long polling can have complex message ordering problems. You have completely different mechanisms for message passing from client-to-server and server-to-client. And middle boxes can and will stall long polled connections, stopping incremental message delivery. (Or you can use one http query per message - but that’s insanely inefficient over the wire).
Websockets are simply a better technology. With long polling, the devil is in the details and it’s insanely hard to get those details right in every case.
One of them 2001 was that Netscape didn't render correctly if the connection is still open. Hah. I am sure this issue has been fixed a long, long time ago, but perhaps there are other issues.
Nowadays I prefer SSE to long polling and websockets.
The idea is: the client doesn't know that the server has new data before it makes a request. With a very simple SSE the client is told that new data is there then it can request new data separately if it wants. This said, SSE has a few quirks, one of them that on HTTP/1 the connection counts to the maximum limit of 6 concurrent connections per browser and domain, so if you have several tabs, you need a SharedWorker to share the connection between the tabs. But probably this quirk also appllies to long polling and websockets. Another quirk, SSE can't transmit binary data and has some limitations in the textual data it represents. But for this use case this doesn't matter.
I would use websockets only if you have a real bidirectional data flow or need to transmit complex data.
WebSocket solves a very different problem. It may be only partially related to organizing two-way communication, but it has nothing to do with data complexity. Moreover, WS are not good enough at transmitting binary data.
If you are using SSE and SW and you need to transfer some binary data from client to server or from server to client, the easiest solution is to use the Fetch API.
`fetch()` handles binary data perfectly well without transformations or additional protocols.
If the data in SW is large enough to require displaying the progress of the data transfer to the server, you will probably be more suited to `XMLHttpRequest`.
> if you have several tabs, you need a SharedWorker to share the connection between the tabs.
You don't have to use a SharedWorker, you can also do domain sharding. Since the concurrent connection limit is per domain, you can add a bunch of DNS records like SSE1.example.org -> 2001:db8::f00; SSE2.example.org -> 2001:db8::f00; SSE3.example.org -> 2001:db8::f00; and so on. Then it's just a matter of picking a domain at random on each page load. A couple hundred tabs ought to be enough for anyone ;)
Good idea if you host a server, but then probably you also could employ HTTP/2 which doesn't have this limitation.
If you offer an executable where the end-user runs it on their own laptop or desktop, you can't expect them to configure multiple domains, but probably there's a way to work-around that: have multiple localhosts like 127.0.0.1, 127.0.0.2, and so on, but there's still one limitation: if the end-user wants to run it on a home server, this won't work as well. In that case tell him to define multiple names for the home server. Ugh.
you can use one http query per message - but that’s insanely inefficient over the wire
Use one http response per message queue snapshot. Send no more than N messages at once. Send empty status if the queue is empty for more than 30-60 seconds. Send cancel status to an awaiting connection if a new connection opens successfully (per channel singleton). If needed, send and accept "last" id/timestamp. These are my usual rules for long-polling.
You can certainly you can do all that. You also need to handle retransmission. And often you also need a way for the client to send back confirmations that each side received certain messages. So, as well as sequence numbers like you mentioned, you probably want acknowledgement numbers in messages too. (Maybe - it depends on the application).
Implementing a stable, in-order, exactly once message delivery system on top of long polling starts to look a lot like implementing TCP on top of UDP. Its a solvable problem. I've done it - 14 years ago I wrote the first opensource implementation of (the server side) of google's Browserchannel protocol, from back before websockets existed:
This supports long polling on browsers, all the way back to IE5.5. It works even when XHR isn't available! I wrote it in literate coffeescript, from back when that was a thing.
But getting all of those little details right is really very difficult. Its a lot of code, and there are a lot of very subtle bugs lurking in this kind of code if you aren't careful. So you also need good, complex testing. You can see in that repo - I ended up with over 1000 lines of server code+comments (lib/server.coffee), and 1500 lines of testing code (test/server.coffee).
And once you've got all that working, my implementation really wanted server affinity. Which made load balancing & failover across over multiple application servers a huge headache.
It sounds like your application allows you to simplify some details of this network protocol code. You do you. I just use websockets & server-sent events. Let TCP/IP handle all the details of in-order message delivery. Its really quite good.
This is a common library issue, it doesn’t know and has to be defensive and featureful at the same time.
Otoh, end-user projects usually know things and can make simplifying decisions. These two are incomparable. I respect the effort, but I also think that this level of complexity is a wrong answer to the call in general. You have to price-break requirements because they tend to oversell themselves and rarely feature-intersect as much as this library implies. Iow, when a client asks for guarantees, statuses or something we just tell them to fetch from a suitable number of seconds ago and see themselves. Everyone works like this, you need some extra - track it yourself based on your own metrics and our rate limits.
Here it's easy to specify that a endpoint is a "stream", and then the code-generation tool gives all tools really to just keep serving the client with multiple responses. It looks deceptively simple. We already have setup auth, logging and metrics for gRPC, so I hope it just works off of that maybe with minor adjustments.
But I'm guessing you don't need the gRPC layer to use HTTP/2 Multiplexing?
At least in a browser context, HTTP/2 doesn't address server to client unsolicitied messages. So you'd still need a polling request open from the client.
HTTP/2 does specify a server push mechanism (PUSH_PROMISE), but afaik, browsers don't accept them and even if they did, (again afaik) there's no mechanism for a page to listen for them.
But if you control the client and the server, you could use it.
gRPC as specced to ride directly on top of HTTP/2 doesn't work from browsers, the sandboxed JS isn't allowed that level of control over the protocol. And often is too low level to implement as part of a pre-existing HTTP server, too. gRPC is a server-to-server protocol that is not part of the usual Web, but happens to repurpose HTTP/2.
Outside of gRPC, just HTTP POST cannot at this time replace websockets because the in-browser `fetch` API doesn't support streaming request body. For now, websockets is the only thing that can natively provide an ordered stream of messages from browser to server.
I don't know how meaningful it is any more, but with long polling with a short timeout and a gracefully ended request (i.e. chunked encoding with an eof chunk sent rather than disconnection), the browser would always end up with one spare idle connection to the server, making subsequent HTTP requests for other parts of the UI far more likely to be snappier, even if the app has been left otherwise idle for half the day
I guess at least this trick is still meaningful where HTTP/2 or QUIC aren't in use
hah seriously. my app uses web sockets extensively but since we are also using Phoenix, its never been source of conflict in development. it really was just drop it and scale to thousands of users.
to clarify, it does scale better out of the box. a clustered websocket setup where reconnections can go to a different machine and resume state works our of the box. its a LOT of work do do that in nodejs. I've done both.
The icing on the cake is that you can also enable Phoenix channels to fallback to longpolling in your endpoint config. The generator sets it to false by default.
It is also fire and forget with fall over to http if web sockets aren't available. I believe if web sockets don't work it can fall over to http long polling instead, but don't quote me on that.
All the downsides of web sockets mentioned in the article are handled for you. Plus you can re-use your existing auth solution. Easily plug logging stuff in, etc. etc. Literally all the problems mentioned by the author are dealt with.
Given the author mentions C# is part of their stack I don't know why they didn't mention signalr or use that instead of rolling their own solution.
To further clarify, I believe Blazor uses SignalR under-the-hood when doing server-side rendering. So the direct answer is probably "this is similar to a component used in Blazor"
Edit: Whoops, I lost context here. Phoenix LiveView as a whole is probably pretty analogous to Blazor.
>Given the author mentions C# is part of their stack I don't know why they didn't mention signalr or use that instead of rolling their own solution.
Not saying this is why but SignalR is notoriously buggy. I've never seen a real production instance that didn't have issues. I say that as someone who probably did one of the first real world, large scale roll outs of SignalR about a decade ago.
I believe it's changed alot since then? I was abit cynical of it from that initial experience but recent usage of it made it seem reliable and scalable.
We use it in a production system on the aspnetcore stack, and it's been bulletproof for a number of years.
We have lots of proxies in the way, and old tech on certain networks, and at this moment we are 97% websockets, 2% server side events, 1% long polling, and it's all transparent to me, brilliant.
I am using Blazor Server, and for some reason a server is not allowing Web Socket connections(troubleshooting this) and the app switches to long polling as fallback.
Phoenix predates Blazer, but they are both server-side rendered frameworks.
In terms of real time updates, Phoenix typically relies on LiveView, which uses web sockets and falls back to long-polling if necessary. I think SignalR is the closest equivalent in the .Net world.
every websocket setup is painless when running on a single server or handling very few connections...
I was on the platform/devops/man w/ many hats team for an elixir shop running Phoenix in k8s. WS get complicated even in elxir when you have 2+ app instances behind a round robin load balancer. You now need to share broadcasts between app servers. Here's a situation you have to solve for w/ any app at scale regardless of language
app server #1 needs to send a publish/broadcast message out to a user, but the user who needs that message isn't connected to app server #1 that generated the message, that user is currently connected to app server #2.
How do you get a message from one app server to the other one which has the user's ws connection?
A bad option is sticky connections. User #1 always connects to server #1. Server #1 only does work for users connected to it directly. Why is this bad? Hot spots. Overloaded servers. Underutilized servers. Scaling complications. Forecasting problems. Goes against the whole concept of horizontal scaling and load balancing. It doesn't handle side-effect messages, ie user #1000 takes some action which needs to broadcast a message to user #1 which is connected to who knows where.
The better option: You need to broadcast to a shared broker. Something all app servers share a connection to so they can themselves subscribe to messages they should handle, and then pass it to the user's ws connection. This is a message broker. postgres can be that broker, just look at oban for real world proof. Throw in pg's listen/notify and you're off to the races. But that's heavy from a resources per db conn perspective so lets avoid the acid db for this then. Ok. Redis is a good option, or since this is elixir land, use the built in distributed erlang stuff. But, we're not running raw elixir releases on linux, we're running inside of containers, on top of k8s. The whole distributed erlang concept goes to shit once the erlang procs are isolated from each other and not in their perfect Goldilocks getting started readme world. So ok, in containers in k8s, so each app server needs to know about all the other app servers running, so how do you do that? Hmm, service discovery! Ok, well, k8s has service discovery already, so how do I tell the erlang vm about the other nodes that I got from k8s etcd? Ah, a hex package cool. lib_cluster to the rescue https://github.com/bitwalker/libcluster
So we'll now tie the boot process of our entire app to fetching the other app server pod ips from k8s service discovery, then get a ring of distributed erlang nodes talking to each other, sharing message passing between them, this way no matter which server the lb routes the user to, a broadcast from any one of them will be seen by all of them, and the one who holds the ws connection will then forward it down the ws to the user.
So now there's a non trivial amount of complexity and risk that was added here. More to reason about when debugging. More to consider when working on features. More to understand when scaling, deploying, etc. More things to potentially take the service down or cause it not to boot. More things to have race conditions, etc.
Nothing is ever so easy you don't have to think about it.
Were the servers part of an Elixir cluster? Because that functionality should be transparent. It sounds to me like you had them set up as two independent nodes that were not aware of each other.
Yes, and I mentioned redis above. You need a message broker with 2+ servers, be it pg, redis, or distributed erlang. It's still more complex than a single server setup, which was my point. It's easy to say you don't have to think about things working when you have 1 server, it's not easy to say that past 1 server, because the complexity added changes that picture.
The full schema isn’t listed, but the indices don’t make sense to me.
(id, cluster_id) sounds like it could / should be the PK
If the jobs are cleared once they’ve succeeded, and presumably retried if they’ve failed or stalled, then the table should be quite small; so small, that a. The query planner is unlikely to use the partial index on (status) b. The bloat from the rapidity of DELETEs likely overshadows the live tuple size.
I implemented a long polling solution in desktop software over 20 years ago and it’s still working great. It can even be used as a tunnel to stream RDP sessions, through which YouTube can play without a hiccup. Big fan of long polling, though I admit I didn’t get a chance to try web sockets back then.
i can't speak for why the author chose it, but if you're operating behind AWS cloudfront then http requests have a maximum timeout of 60s - if you don't close the request within 60s, cloudfront will close it for you.
i suspect other firewalls, cdns, or reverse proxy products will all do something similar. for me, this is one of the biggest benefits of websockets over long-polling: it's a standard way to communicate to proxies and firewalls "this connection is supposed to stay open, don't close it on me"
I don't think any of the methods give any significant advantage since in the end you need to maintain a connection per each client. The difference between the methods boils down to complexity of implementation and reliability.
If you want to reduce server load then you'd have to sacrifice responsiveness, e.g. you perform short polls at certain intervals, say 10s.
> Corporate firewalls blocking WebSocket connections was one of our other worries. Some of our users are behind firewalls, and we don't need the IT headache of getting them to open up WebSockets.
Don't websockets look like ordinary https connections?
AFAIK, only by symptoms. If https fetches work and websockets don't, that's a sign. HSTS and assorted reporting can help a bit in aggregate, but not if the corporate MITM CA has been inserted into the browser's trusted CA list. I don't think there's an API to get certificate details from the browser side to compare.
A proxy may have a different TLS handshake than a real browser would, depending on how good the MITM is, but the better they are, the more likely it is that websockets work.
I think this article is tying a lot of unrelated decisions to "Websocket" vs "Long-polling" when they're actually independent. A long-polling server could handle a websocket client with just a bit of extra work to handle keep-alive.
For the other direction, to support long-polling clients if your existing architecture is websockets which get data pushed to them by other parts of the system, just have two layers of servers: one which maintains the "state" of the connection, and then the HTTP server which receives the long polling request can connect to the server that has the connection state and wait for data that way.
Having done this, I don't think I'd reduce it to "just a little bit of work" to make it hum in production.
Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world.
> Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world.
How so? As a minimal change, the thing on the server end of the websocket could just do the polling of your database on its own while the connection is open (using authorization credentials supplied as the websocket is being opened). If the connection dies, stop polling. This has the nice property that you're in full control of the refresh rate, can implement coordinated backoffs if the database is overloaded, etc.
Yes, but that's one part of it though. For example, you have to:
- change how you hydrate the initial state in the web component.
- rework any request-oriented configurations you do at the edge based on the payloads. (For example, if you use cloudflare and use their HTTP rules, you have to rework that)
I would appreciate if the article spent more time actually discussing the benefits of websockets (and/or more modern approaches to pushing data from server -> browser) and why the team decided those benefits were not worth the purported downsides. I could see the same simplicity argument being applied to using unencrypted HTTP/1.1 instead of HTTP/2, or TCP Reno instead of CUBIC.
The section at the end talking about "A Case for Websockets" really only rehashes the arguments made in "Hidden Benefits of Long-Polling" stating that you need to reimplement these various mechanisms (or just use a library for it).
My experience in this space is from 2011, when websockets were just coming onto the scene. Tooling / libraries were much more nascent, websockets had much lower penetration (we still had to support IE6 in those days!), and the API was far less stable prior to IETF standardization. But we still wanted to use them when possible, since they provided much better user experience (lower latency, etc) and lower server load.
The points mentioned against websockets are mostly fud, I've used websockets in production for a very heavy global data streaming application, and I would respond the following to the "upsides" of not using websockets:
> Observability Remains Unchanged
Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either.
> Authentication Simplicity
Sure, auth is different than with http, but not more difficult. You can easily pass a token.
> Infrastructure Compatibility
I'm sure you can find firewalls out there where websockets are blocked, however for my use case I have never seen this reported. I think this is outdated, for sure you don't need "special proxy configurations or complex infrastructure setups".
> Operational Simplicity
Restarts will drop any persistent connection, state can be both or neither in WS or in LP, it doesn't matter what you use.
> Client implementation
It mentions "no special WebSocket libraries needed" and also "It works with any HTTP client". Guess what, websockets will work with any websocket client! Who knew!
Finally, in the conclusion:
> For us, staying close to the metal with a simple HTTP long polling implementation was the right choice
Calling simple HTTP long polling "close to the metal" in comparison to websockets is weird. I wouldn't be surprised if websockets scale much better and give much more control depending on the type of data, but that's besides the point. If you want to use long polling because you prefer it, go ahead. Its a great way to stick to request/response style semantics that web devs are familiar with. Its not necessary to regurgitate a bunch of random hearsay arguments that may influence people in the wrong way.
Try to actually leave the reader with some notion of when to use long polling vs when to use websockets, not a post-hoc justification of your decision based on generalized arguments that do not apply.
> Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either.
As a person who works in a large company handling millions of websockets, I fundamentally disagree with discounting the observability challenges. WebSockets completely transform your observability stack - they require different logging patterns, new debugging approaches, different connection tracking, and change how you monitor system health at scale. Observability is far more than metrics, and handwaving away these architectural differences doesn't make the implementation easier.
> with discounting the observability challenges
> handwaving away these architectural differences
I am doing neither of these things. I am only saying you will have observability problems whether you do LP or WS, because you are stepping away from the request/response model that most tools work with. As such, its weird to argue that "observability remains unchanged".
I think they are mixing some problems here. They could probably have used their original setup with Postgres NOTIFY+triggers in stead of polling, and only have one "pickup poller" to catch any missed events/jobs. In my opinion transaction medium should not be linked to how the data is manage internally, but I know from experience that this separation is often hard to achieve in practice.
The article does discuss a lot of mixed concepts. I would prefer one process polling new jobs/state and one process handling http connections/websockets. Hence no flooding the database and completely scalable from the client side. The database process pushes everything downstream via some queue while the other process/server handles those and sends them to respective clients
Since the article mentioned Postgres by name, isn't this a case for using its asynchronous notification features? Servers can LISTEN to a channel and PG can TRIGGER and NOTIFY them when the data changes.
No polling needed, regardless of the frontend channel.
It would be easier to run Hypermode’s Dgraph as the database and use GraphQL subscriptions from the frontend. But nobody ever got fired for choosing postgres.
In the context of Node.js, where op said, yes it is easier. But it's a new thing and most people don't realize timers in Node are awaitable yet, so the other way is less about "works everywhere" and more "this is just what I know"
I guess most Node.js developers also don't realize that there's "node:fs/promises" so you don't have to use callbacks or manually wrap functions from "node:fs" with util.promisify(). Doesn't mean need to stick with old patterns forever.
When I said 'in the context of Node.js' I meant if you are in a JS module where you already import other node: modules, ie. when it's clear that code runs in a Node.js runtime and not in a browser. Of course when you are writing code that's supposed to be portable, don't use it. Or don't use setTimeout at all because it's not guaranteed to be available in all runtimes - it's not part of the ECMA-262 language specification after all.
Neither Server-Sent Events nor WebSockets have replaced all use cases of long polling reliably. The connection limit of SSE comes up a lot, even if you’re using HTTP/2. WebSockets, on the other hand, are unreliable as hell in most environments. Also, WS is hard to debug, and many of our prod issues with WS couldn’t even be reproduced locally.
Detecting changes in the backend and propagating them to the right client is still an unsolved problem. Until then, long polling is surprisingly simple and a robust solution that works.
> The connection limit of SSE comes up a lot, even if you’re using HTTP/2.
I'm considering using SSE for an app. I'm curious, what problems you've run into? At least the docs say you get 100 connections between the server and a client, but it can be negotiated higher if needed it seems?
SSEs are great and more reliable than websockets in a smaller scale. So I'd reach for it despite the issues. But that being said, some websevers don't play well with SSE and you'll need to fiddle with it. If you control the webserver, then it's not much of a problem.
172 comments
[ 2.8 ms ] story [ 245 ms ] threadBoth are similar in that they hold the HTTP connection open and have the benefit of being simply HTTP (the big plus here). SSE (at least to me) feels like it's more suitable for some use cases where updates/results could be streamed in.
A fitting use case might be where you're monitoring all job IDs on behalf of a given client. Then you could move the job monitoring loop to the server side and continuously yield results to the client.
I've not personally witnessed this, but people on the internets have said that _some_ proxies/LBs have problems with SSE due to the way it does buffering.
I am curious about what you mean here. The 'text/event-stream' allows for abitrary event formats, it just provides structure for EventSource to be able to parse.
You should only need one 'text/event-stream' and should be able send the same JSON via normal or SSE response.
One of the drawbacks, as I learned - SSE have limit on number of up to ~6 open connections (browser + domain name). This can quickly become a limiting factor when you open the same web page in multiple tabs.
The big downside of sse in mobile safari - at least a few years ago - is you got a constant loading spinner on the page. Thats bad UX.
Connections are dropped all the time, and then your code, on both client and server, need to account for retries (will the reconnection use a cached DNS entry? how will load balancing affect long term connections?), potentially missed events (now you need a delta between pings), DDoS protections (is this the same client connecting from 7 IPs in a row or is this a botnet), and so on.
Regular polling great reduces complexity on some of these points.
Yes, WS is complex. Long polling is not much better.
I can’t help but think that if front end connections are destroying your database, then your code is not structured correctly. You can accept both WS and long polls without touching your DB, having a single dispatcher then send the jobs to the waiting connections.
Clients using mobile phones tend to have their IPs rapidly changed in sequence.
I didn't mention databases, so I can't comment on that point.
But websockets also guarantee in-order delivery, which is never guaranteed by long polling. And websockets play way better with intermediate proxies - since nothing in the middle will buffer the whole response before delivering it. So you get better latency and better wire efficiency. (No http header per message).
At this point, long polling seems to carry more benefits, IMHO. WebSockets seem to be excellent for stable conditions, but not quite what we need for mobile.
I don't see how this is meaningfully different for long polling. The client could have received some updates but never ack'd it successfully over a long poll, so either way you need to keep a log and resync on reconnection.
- Observability: WebSockets are more stateful, so you need to implement additional logging and monitoring for persistent connections: solved with graphql if the existing monitoring is already sufficient.
- Authentication: You need to implement a new authentication mechanism for incoming WebSocket connections: solved with graphql.
- Infrastructure: You need to configure your infrastructure to support WebSockets, including load balancers and firewalls: True, firewalls need to be updated.
- Operations: You need to manage WebSocket connections and reconnections, including handling connection timeouts and errors: normally already solved by the graphql library. For errors, it's basically the same though.
- Client Implementation: You need to implement a client-side WebSocket library, including handling reconnections and state management: Just have to use a graphql library that comes with websocket support (I think most of them do) and configure it accordingly.
I vomit in my mouth a bit whenever people reach for socket.io or junk like that. You don’t want or need the complexity and bugs these libraries bring. They’re obsolete.
As a workaround in one project I wrote JavaScript code which manually sent cookies in the first websocket message from the client as soon as a connection opened. But I think this problem is now solved in all browsers.
One thing that seems clumsy in the code example is the loop that queries the data again and again. Would be nicer if the data update could also resolve the promise of the response directly.
Websockets are simply a better technology. With long polling, the devil is in the details and it’s insanely hard to get those details right in every case.
Nowadays I prefer SSE to long polling and websockets.
The idea is: the client doesn't know that the server has new data before it makes a request. With a very simple SSE the client is told that new data is there then it can request new data separately if it wants. This said, SSE has a few quirks, one of them that on HTTP/1 the connection counts to the maximum limit of 6 concurrent connections per browser and domain, so if you have several tabs, you need a SharedWorker to share the connection between the tabs. But probably this quirk also appllies to long polling and websockets. Another quirk, SSE can't transmit binary data and has some limitations in the textual data it represents. But for this use case this doesn't matter.
I would use websockets only if you have a real bidirectional data flow or need to transmit complex data.
Server-sent events.
If you are using SSE and SW and you need to transfer some binary data from client to server or from server to client, the easiest solution is to use the Fetch API. `fetch()` handles binary data perfectly well without transformations or additional protocols.
If the data in SW is large enough to require displaying the progress of the data transfer to the server, you will probably be more suited to `XMLHttpRequest`.
You don't have to use a SharedWorker, you can also do domain sharding. Since the concurrent connection limit is per domain, you can add a bunch of DNS records like SSE1.example.org -> 2001:db8::f00; SSE2.example.org -> 2001:db8::f00; SSE3.example.org -> 2001:db8::f00; and so on. Then it's just a matter of picking a domain at random on each page load. A couple hundred tabs ought to be enough for anyone ;)
If you offer an executable where the end-user runs it on their own laptop or desktop, you can't expect them to configure multiple domains, but probably there's a way to work-around that: have multiple localhosts like 127.0.0.1, 127.0.0.2, and so on, but there's still one limitation: if the end-user wants to run it on a home server, this won't work as well. In that case tell him to define multiple names for the home server. Ugh.
Use one http response per message queue snapshot. Send no more than N messages at once. Send empty status if the queue is empty for more than 30-60 seconds. Send cancel status to an awaiting connection if a new connection opens successfully (per channel singleton). If needed, send and accept "last" id/timestamp. These are my usual rules for long-polling.
Prevents: connection overhead, congestion latency, connection stalling, unwanted multiplexing, sync loss, respectively.
You have completely different mechanisms for message passing from client-to-server and server-to-client
Is this a problem? Why should this even be symmetric?
Implementing a stable, in-order, exactly once message delivery system on top of long polling starts to look a lot like implementing TCP on top of UDP. Its a solvable problem. I've done it - 14 years ago I wrote the first opensource implementation of (the server side) of google's Browserchannel protocol, from back before websockets existed:
https://github.com/josephg/node-browserchannel
This supports long polling on browsers, all the way back to IE5.5. It works even when XHR isn't available! I wrote it in literate coffeescript, from back when that was a thing.
But getting all of those little details right is really very difficult. Its a lot of code, and there are a lot of very subtle bugs lurking in this kind of code if you aren't careful. So you also need good, complex testing. You can see in that repo - I ended up with over 1000 lines of server code+comments (lib/server.coffee), and 1500 lines of testing code (test/server.coffee).
And once you've got all that working, my implementation really wanted server affinity. Which made load balancing & failover across over multiple application servers a huge headache.
It sounds like your application allows you to simplify some details of this network protocol code. You do you. I just use websockets & server-sent events. Let TCP/IP handle all the details of in-order message delivery. Its really quite good.
Otoh, end-user projects usually know things and can make simplifying decisions. These two are incomparable. I respect the effort, but I also think that this level of complexity is a wrong answer to the call in general. You have to price-break requirements because they tend to oversell themselves and rarely feature-intersect as much as this library implies. Iow, when a client asks for guarantees, statuses or something we just tell them to fetch from a suitable number of seconds ago and see themselves. Everyone works like this, you need some extra - track it yourself based on your own metrics and our rate limits.
I have only tried it briefly when we use gRPC: https://grpc.io/docs/what-is-grpc/core-concepts/#server-stre...
Here it's easy to specify that a endpoint is a "stream", and then the code-generation tool gives all tools really to just keep serving the client with multiple responses. It looks deceptively simple. We already have setup auth, logging and metrics for gRPC, so I hope it just works off of that maybe with minor adjustments. But I'm guessing you don't need the gRPC layer to use HTTP/2 Multiplexing?
HTTP/2 does specify a server push mechanism (PUSH_PROMISE), but afaik, browsers don't accept them and even if they did, (again afaik) there's no mechanism for a page to listen for them.
But if you control the client and the server, you could use it.
Outside of gRPC, just HTTP POST cannot at this time replace websockets because the in-browser `fetch` API doesn't support streaming request body. For now, websockets is the only thing that can natively provide an ordered stream of messages from browser to server.
I guess at least this trick is still meaningful where HTTP/2 or QUIC aren't in use
It is also fire and forget with fall over to http if web sockets aren't available. I believe if web sockets don't work it can fall over to http long polling instead, but don't quote me on that.
All the downsides of web sockets mentioned in the article are handled for you. Plus you can re-use your existing auth solution. Easily plug logging stuff in, etc. etc. Literally all the problems mentioned by the author are dealt with.
Given the author mentions C# is part of their stack I don't know why they didn't mention signalr or use that instead of rolling their own solution.
Edit: Whoops, I lost context here. Phoenix LiveView as a whole is probably pretty analogous to Blazor.
Not saying this is why but SignalR is notoriously buggy. I've never seen a real production instance that didn't have issues. I say that as someone who probably did one of the first real world, large scale roll outs of SignalR about a decade ago.
We have lots of proxies in the way, and old tech on certain networks, and at this moment we are 97% websockets, 2% server side events, 1% long polling, and it's all transparent to me, brilliant.
I am using Blazor Server, and for some reason a server is not allowing Web Socket connections(troubleshooting this) and the app switches to long polling as fallback.
In terms of real time updates, Phoenix typically relies on LiveView, which uses web sockets and falls back to long-polling if necessary. I think SignalR is the closest equivalent in the .Net world.
I was on the platform/devops/man w/ many hats team for an elixir shop running Phoenix in k8s. WS get complicated even in elxir when you have 2+ app instances behind a round robin load balancer. You now need to share broadcasts between app servers. Here's a situation you have to solve for w/ any app at scale regardless of language
app server #1 needs to send a publish/broadcast message out to a user, but the user who needs that message isn't connected to app server #1 that generated the message, that user is currently connected to app server #2.
How do you get a message from one app server to the other one which has the user's ws connection?
A bad option is sticky connections. User #1 always connects to server #1. Server #1 only does work for users connected to it directly. Why is this bad? Hot spots. Overloaded servers. Underutilized servers. Scaling complications. Forecasting problems. Goes against the whole concept of horizontal scaling and load balancing. It doesn't handle side-effect messages, ie user #1000 takes some action which needs to broadcast a message to user #1 which is connected to who knows where.
The better option: You need to broadcast to a shared broker. Something all app servers share a connection to so they can themselves subscribe to messages they should handle, and then pass it to the user's ws connection. This is a message broker. postgres can be that broker, just look at oban for real world proof. Throw in pg's listen/notify and you're off to the races. But that's heavy from a resources per db conn perspective so lets avoid the acid db for this then. Ok. Redis is a good option, or since this is elixir land, use the built in distributed erlang stuff. But, we're not running raw elixir releases on linux, we're running inside of containers, on top of k8s. The whole distributed erlang concept goes to shit once the erlang procs are isolated from each other and not in their perfect Goldilocks getting started readme world. So ok, in containers in k8s, so each app server needs to know about all the other app servers running, so how do you do that? Hmm, service discovery! Ok, well, k8s has service discovery already, so how do I tell the erlang vm about the other nodes that I got from k8s etcd? Ah, a hex package cool. lib_cluster to the rescue https://github.com/bitwalker/libcluster
So we'll now tie the boot process of our entire app to fetching the other app server pod ips from k8s service discovery, then get a ring of distributed erlang nodes talking to each other, sharing message passing between them, this way no matter which server the lb routes the user to, a broadcast from any one of them will be seen by all of them, and the one who holds the ws connection will then forward it down the ws to the user.
So now there's a non trivial amount of complexity and risk that was added here. More to reason about when debugging. More to consider when working on features. More to understand when scaling, deploying, etc. More things to potentially take the service down or cause it not to boot. More things to have race conditions, etc.
Nothing is ever so easy you don't have to think about it.
Elixir gives more options and lets you do it natively.
Also, there are simpler options for clustering out there like https://github.com/phoenixframework/dns_cluster (Disclaimer: I am a contributor)
Anyway I agree that once you go with more than one server it's a whole new world but not sure if it's easier in any other language.
(id, cluster_id) sounds like it could / should be the PK
If the jobs are cleared once they’ve succeeded, and presumably retried if they’ve failed or stalled, then the table should be quite small; so small, that a. The query planner is unlikely to use the partial index on (status) b. The bloat from the rapidity of DELETEs likely overshadows the live tuple size.
i suspect other firewalls, cdns, or reverse proxy products will all do something similar. for me, this is one of the biggest benefits of websockets over long-polling: it's a standard way to communicate to proxies and firewalls "this connection is supposed to stay open, don't close it on me"
What's the most resource efficient way to push data to clients over HTTP?
I can send data to a server via HTTP request, I just need a way to notify a client about a change and would like to avoid polling for it.
I heard talk about SSE, WebSockets, and now long-polling.
Is there something else?
What requires the least resources on the server?
If you want to reduce server load then you'd have to sacrifice responsiveness, e.g. you perform short polls at certain intervals, say 10s.
What's the least complex to implement then?
For other clients, such as mobile apps, I think long poll would be the simplest.
Don't websockets look like ordinary https connections?
A proxy may have a different TLS handshake than a real browser would, depending on how good the MITM is, but the better they are, the more likely it is that websockets work.
For the other direction, to support long-polling clients if your existing architecture is websockets which get data pushed to them by other parts of the system, just have two layers of servers: one which maintains the "state" of the connection, and then the HTTP server which receives the long polling request can connect to the server that has the connection state and wait for data that way.
Personally I would enjoyed solving that problem instead of hacking around it but that’s me.
Having done this, I don't think I'd reduce it to "just a little bit of work" to make it hum in production.
Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world.
Only that knows url for endpoints, protocols and connections - and proxies between them and your app / components
How so? As a minimal change, the thing on the server end of the websocket could just do the polling of your database on its own while the connection is open (using authorization credentials supplied as the websocket is being opened). If the connection dies, stop polling. This has the nice property that you're in full control of the refresh rate, can implement coordinated backoffs if the database is overloaded, etc.
- change how you hydrate the initial state in the web component. - rework any request-oriented configurations you do at the edge based on the payloads. (For example, if you use cloudflare and use their HTTP rules, you have to rework that)
The section at the end talking about "A Case for Websockets" really only rehashes the arguments made in "Hidden Benefits of Long-Polling" stating that you need to reimplement these various mechanisms (or just use a library for it).
My experience in this space is from 2011, when websockets were just coming onto the scene. Tooling / libraries were much more nascent, websockets had much lower penetration (we still had to support IE6 in those days!), and the API was far less stable prior to IETF standardization. But we still wanted to use them when possible, since they provided much better user experience (lower latency, etc) and lower server load.
> Observability Remains Unchanged
Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either.
> Authentication Simplicity
Sure, auth is different than with http, but not more difficult. You can easily pass a token.
> Infrastructure Compatibility
I'm sure you can find firewalls out there where websockets are blocked, however for my use case I have never seen this reported. I think this is outdated, for sure you don't need "special proxy configurations or complex infrastructure setups".
> Operational Simplicity
Restarts will drop any persistent connection, state can be both or neither in WS or in LP, it doesn't matter what you use.
> Client implementation
It mentions "no special WebSocket libraries needed" and also "It works with any HTTP client". Guess what, websockets will work with any websocket client! Who knew!
Finally, in the conclusion:
> For us, staying close to the metal with a simple HTTP long polling implementation was the right choice
Calling simple HTTP long polling "close to the metal" in comparison to websockets is weird. I wouldn't be surprised if websockets scale much better and give much more control depending on the type of data, but that's besides the point. If you want to use long polling because you prefer it, go ahead. Its a great way to stick to request/response style semantics that web devs are familiar with. Its not necessary to regurgitate a bunch of random hearsay arguments that may influence people in the wrong way.
Try to actually leave the reader with some notion of when to use long polling vs when to use websockets, not a post-hoc justification of your decision based on generalized arguments that do not apply.
> Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either.
As a person who works in a large company handling millions of websockets, I fundamentally disagree with discounting the observability challenges. WebSockets completely transform your observability stack - they require different logging patterns, new debugging approaches, different connection tracking, and change how you monitor system health at scale. Observability is far more than metrics, and handwaving away these architectural differences doesn't make the implementation easier.
I am doing neither of these things. I am only saying you will have observability problems whether you do LP or WS, because you are stepping away from the request/response model that most tools work with. As such, its weird to argue that "observability remains unchanged".
No polling needed, regardless of the frontend channel.
In terms of not getting fired - Postgres is a lot more innovative than most databases, and the insinuation of IBM.
By innovative I mean uniquely putting in performance related items for the last 10-20 years.
When I said 'in the context of Node.js' I meant if you are in a JS module where you already import other node: modules, ie. when it's clear that code runs in a Node.js runtime and not in a browser. Of course when you are writing code that's supposed to be portable, don't use it. Or don't use setTimeout at all because it's not guaranteed to be available in all runtimes - it's not part of the ECMA-262 language specification after all.
I just don't see the point. It doesn't work in the browser and it shadows global.setTimeout which is confusing. Meanwhile the idiom works everywhere.
To me it's kinda like adding a shallowClone(old) helper instead of writing const obj = { ...old }.
But no point in arguing about it forever.
https://socket.io/docs/v4/how-it-works/#upgrade-mechanism
Detecting changes in the backend and propagating them to the right client is still an unsolved problem. Until then, long polling is surprisingly simple and a robust solution that works.
I'm considering using SSE for an app. I'm curious, what problems you've run into? At least the docs say you get 100 connections between the server and a client, but it can be negotiated higher if needed it seems?
https://developer.mozilla.org/en-US/docs/Web/API/EventSource