Yeah! these are just normal websockets. We're definitely interested in supporting and testing more protocols and ways of accessing postgres and neon as we can. We're also super excited to see the development and adoption of new protocols!
The axis on all your graphs are unlabeled and have the same scale in both the X and the Y, accordingly I am not sure exactly what is being shown on the graph.
Fair point! We'll fix this. In the meantime, the graphs show latency distributions, so the X axis is time (in milliseconds) and the Y axis is a simple count.
Pardon my ignorance as I'm not aware of these recent developments, but what is a valid use case for this? Why would I put my database into an ephemeral serverless DB[1]?
Well, the data is absolutely not ephemeral: it's safely tucked away in S3.
But to crib from https://neon.tech, you might choose Neon because it's scalable, cost-effective, and easy to deploy and manage. Or you might choose it for its copy-on-write branching and point-in-time recovery, all open-source, and all on top of the solid, performant, feature-rich foundation that is Postgres. :)
Neon separates storage and compute. The compute part is serverless and can scale up and down on demand. The storage has multiple layers: Safekeepers for durability and Pageservers to reconstruct pages from WAL.
What’s the difference in performance between websockets and http with large datasets. I would imagine http might be faster? Do you encode your data with something like messagepack when using web sockets?
@raoufchebri & @nikita, What about comparisons to CloudFlare Workers using Workers Sock API to connect to PG? [1] I know CloudFlare workers support Neon directly through API so how does that compare to the alternative PG implementations?
It seems like if speed was a concern you wouldn't make the extra hop from a serverless function to their websocket server, then on to the db. Instead you could serve your api directly off the app server which would then connect to the db without the intermediary step.
Every connection to a Neon database (traditional TCP, WebSockets, https) now goes via the same custom (open-source) proxy server that's close to the database compute node.
So there's technically an additional hop irrespective of the transport type, but not one that should add meaningfully to the latency, or that would make WebSockets slower than the others.
The diagram is confusing me too. We are using Az Functions to serve webapps via HTTP triggers and they just take a direct connection to SQL. Our average trip to the database is ~5 milliseconds for the trivial views. No web sockets, middleware, etc. What value is added with this extra middleware/ceremony/etc?
Right. If TCP is available, any common-or-garden Postgres driver will do (for Neon or any other Postgres DB). But not all serverless compute environments support TCP connections.
> Our average trip to the database is ~5 milliseconds
Do you miss the performance that you had when the DB was running inside the application process using SQLite? Are you at all sad that you had to give that up in order to outsource compliance to Azure?
> Are you at all sad that you had to give that up in order to outsource compliance to Azure?
A little bit, but it's honestly much better this way in our use case. For our specific business & application, going from 50uS to 5000uS feels like nothing. We were squandering 2+ orders of magnitude over JSON blobs in columns anyways. We really didn't need 50uS transactions for anything. What turned out to be far more important was one source of truth to worry about. We have a lot of installs and a lot of SQLite databases out there. Now we are looking at exactly one for the whole enterprise.
Moving to hosted DB forced cleaner schema design. No json blobs = law now because of IO scaling concerns. We can't afford to move 5 megabytes of JSON across the datacenter every time someone clicks a button anymore. Average per-user-action database hit has fallen by 4~5 orders of magnitude. Megabytes to hundreds of bytes. Overall perceived performance is actually much better because of these design constraints. We also try to get the web views composed using as few queries as possible (i.e. finally make the database do its damn job instead of lazily peddling JSON).
What I would absolutely love is to have a proxy server that would act as postgres, and use this new driver to connect to a remote postgres. This would avoid us having to rewrite our remote agents (which require a postgres connection to the main Database) to use an http api. Instead, we could just keep them as it is, using rust's sqlx queries, point them to the local proxy. Essentially pgBouncer but through http.
Postgres <> Neon HTTP Proxy <---HTTP---> Local Fake Postgres Server <--unix socket--> Agents
We need it for the open-source project windmill.dev and we will build this if no one else does it.
This sounds cool (but it's not something we've built at Neon).
In case it's useful, I can clarify:
* Every Neon database is accessible over an ordinary TCP Postgres connection (as well as WebSockets and http).
* Our JavaScript serverless driver is based on node-postgres and is a drop-in replacement for it. So for JS, no rewriting is needed.
Assuming that doesn't help, do you specifically need http (rather than WebSockets), and a proxy that speaks the pg protocol?
If not, the simplest thing might be to (1) use Neon, or run a WebSocket proxy next to your own database (e.g. https://github.com/neondatabase/wsproxy); and (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
Windmill is written in Rust so cannot use the javascript serverless driver. I love sqlx and sqlx would not be easy to replace so short of making it an extension for sqlx, the only route is to make believe the sqlx connection pool that it's talking to a pg server, when it's actually just talking to the local proxy.
If we were using javascript/typescript, then yes we would probably just be able to use that library.
> (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
It's that part that I would love to see. My assumption is that this proxy next to my agents would need to speak pg protocol and then transcribe it to wss. If there is a simpler way, let me know :)
You don't mention which version of HTTP you were using, which makes some of my comments here conjecture, but if I had to guess, I'd say you were using HTTP/1.1. Is that right?
It would really help if you could break down where the time is being spent. I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc.
Did you consider HTTP/2 or 3? I would expect them to have lower setup overhead, and be more competitive with Websockets.
In particular, if you were using HTTP/2+, did you look into CONNECT proxying? It could replace Websockets while having lower connection setup overhead, albeit at the cost of a little payload unwrapping overhead in the proxy. But theoretically, this could offer both the faster startup of HTTP/2+, combined with the lower processing overhead for the db of a TCP stream. Comparing the graphs of 10 queries, it _appears_ that processing HTTP imposes a ~15 ms overhead over raw bytes. (Theoretically, the CONNECT proxy has to obey the host and port in the :authority pseudoheader, but in your case, the proxy would just ignore it, and hook up the edge function to whatever compute node it pleased.)
That's good to rule out db differences, but that doesn't explain where in transit the 15 ms difference is between HTTP and WS for established connections.
I'm assuming the most likely difference is in Pg processing an HTTP request over processing raw bytes, but there are other explanations, too. (Older/slower code pathway for HTTP in proxy, different intermediate node behavior for HTTP vs TCP from proxy to db, etc. Much less likely, but not impossible.)
Yes, exploring http/3 is on the roadmap. One of the learnings for us was the fact that v8 is quite good with caching open TCP connection; For consecutive queries the same TCP+TLS session is being reused. Even when queries are sent from unrelated V8 isolates.
Are you saying that V8 is reusing TCP connections between different browser tabs and websites? Seems like a timing/side-channel attack could be done with that.
In that case, HTTP/3 might actually perform worse for you than HTTP/2, unless v8 caches QUIC conns as well.
The major advantage you'd get with HTTP/3 and QUIC is fewer round trips to start a conn, and no head-of-line blocking like with TCP (which might be irrelevant if the Pg protocol doesn't use simultaneous streams, dunno).
The disadvantage is you'd have to support QUIC in your proxy (and gmac said your server lib doesn't support it yet.)
Don't get hung up on HTTP/3 as the latest and greatest, HTTP/2 is probably still an improvement for you.
> Even when queries are sent from unrelated V8 isolates.
Heh, I've definitely written a few AWS Lambdas that exploited this sort of behavior. "Sure, this resource might not exist. But I'm going to check, just in case..."
Yes, we're currently on HTTP/1.1, but I'm sure we'll be looking at later versions. Unfortunately the Rust web server library we're using doesn't currently support HTTP/3, but we could potentially e.g. put an nginx proxy in front of it for this purpose.
A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
As toomim also replied, HTTP/2 is fine, and honestly, is probably preferable for your use case, since you won't also have to deal with QUIC/UDP, and can keep using TCP.
(HTTP/3 is ~90% the same as HTTP/2, just over a different transport layer, and ditching HTTP/2 prioritization mechanisms which you probably won't care about.)
> A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
Based on the network diagram in your article, it looks like the edge functions speak HTTP to the proxy, upgrade that to WS, and then the proxy routes raw TCP to the conn pool/db.
If so, CONNECT would be the same. The edge function sends an HTTP CONNECT request, the proxy would set it up, and afterwards, both hops are using the same transports as with websockets. The edge function would send HTTP/2 DATA frames wrapping the raw bytes, the proxy would unwrap those bytes, and then route the bytes over TCP to the conn pool/db.
If I understand your setup correctly, HTTP/2 CONNECT would be applicable everywhere WS is.
Neon is slow and flaky I am thankful you are making changes but strongmanning your technical choice when you are wrong is concerning as a user. I still use Neon due to not being able to move back to heroku. I also you keep making open source more.
Performant streaming of structured telemetry data from Edge networks. Edge in this case meaning on-prem installations, not necessarily from a remote DC. These networks have limited network bandwidth and potentially intermittent connectivity, minimizing the message size and overhead as much as possible is ideal.
So you can do a TLS/postgres connection instead of adding additional layers.
Sure you want to probably run a patched pg client/server which has various optimizations to reduce the number of round trips when opening a connection, but that also applies to any HTTP/WebSocket wrapper.
> So you can do a TLS/postgres connection instead of adding additional layers.
Sure, sometimes you can. But a key use-case for a serverless database is connecting from a serverless compute environment, and not all serverless compute environments support TCP connections.
That's why Neon offers this serverless driver, which supports WebSockets and now also http.
Here is the actual package used [1]. I thought it would have been similar to serverless-pg[2] with initiation outside the handler as with the node implementation. Any insights into how they achieve connection caching (similar to pooling?) on the edge?
The biggest selling point of websockets over http is that you can stream the results back easily and quickly, which enables stuff like canceling slow queries easily, a transfer progress bar, etc.
I tried making a realtime multiplayer game network based on HTTP, sending movement data every 100ms and individual world changes on each HTTP requests by that was taking 50ms each on a wifi local network, I thought that was good enough but then I tested websockets and the round-trip for websocket was around 4ms.
49 comments
[ 1.9 ms ] story [ 105 ms ] threadWebSockets still seem to be stuck with TCP as rfc9220 [0] is not implemented I think.
[0]: https://datatracker.ietf.org/doc/rfc9220/
1: https://news.ycombinator.com/item?id=36684121
But to crib from https://neon.tech, you might choose Neon because it's scalable, cost-effective, and easy to deploy and manage. Or you might choose it for its copy-on-write branching and point-in-time recovery, all open-source, and all on top of the solid, performant, feature-rich foundation that is Postgres. :)
The data is also offloaded to S3. Here is a diagram that describes the architecture: https://neon.tech/_next/image?url=https%3A%2F%2Fneon-hwp.dre...
[0]: https://neon.tech/blog/architecture-decisions-in-neon
[1] https://developers.cloudflare.com/workers/learning/integrati...
So there's technically an additional hop irrespective of the transport type, but not one that should add meaningfully to the latency, or that would make WebSockets slower than the others.
Do you miss the performance that you had when the DB was running inside the application process using SQLite? Are you at all sad that you had to give that up in order to outsource compliance to Azure?
A little bit, but it's honestly much better this way in our use case. For our specific business & application, going from 50uS to 5000uS feels like nothing. We were squandering 2+ orders of magnitude over JSON blobs in columns anyways. We really didn't need 50uS transactions for anything. What turned out to be far more important was one source of truth to worry about. We have a lot of installs and a lot of SQLite databases out there. Now we are looking at exactly one for the whole enterprise.
Moving to hosted DB forced cleaner schema design. No json blobs = law now because of IO scaling concerns. We can't afford to move 5 megabytes of JSON across the datacenter every time someone clicks a button anymore. Average per-user-action database hit has fallen by 4~5 orders of magnitude. Megabytes to hundreds of bytes. Overall perceived performance is actually much better because of these design constraints. We also try to get the web views composed using as few queries as possible (i.e. finally make the database do its damn job instead of lazily peddling JSON).
Postgres <> Neon HTTP Proxy <---HTTP---> Local Fake Postgres Server <--unix socket--> Agents
We need it for the open-source project windmill.dev and we will build this if no one else does it.
In case it's useful, I can clarify:
* Every Neon database is accessible over an ordinary TCP Postgres connection (as well as WebSockets and http).
* Our JavaScript serverless driver is based on node-postgres and is a drop-in replacement for it. So for JS, no rewriting is needed.
Assuming that doesn't help, do you specifically need http (rather than WebSockets), and a proxy that speaks the pg protocol?
If not, the simplest thing might be to (1) use Neon, or run a WebSocket proxy next to your own database (e.g. https://github.com/neondatabase/wsproxy); and (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
If we were using javascript/typescript, then yes we would probably just be able to use that library.
> (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
It's that part that I would love to see. My assumption is that this proxy next to my agents would need to speak pg protocol and then transcribe it to wss. If there is a simpler way, let me know :)
You don't mention which version of HTTP you were using, which makes some of my comments here conjecture, but if I had to guess, I'd say you were using HTTP/1.1. Is that right?
It would really help if you could break down where the time is being spent. I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc.
Did you consider HTTP/2 or 3? I would expect them to have lower setup overhead, and be more competitive with Websockets.
In particular, if you were using HTTP/2+, did you look into CONNECT proxying? It could replace Websockets while having lower connection setup overhead, albeit at the cost of a little payload unwrapping overhead in the proxy. But theoretically, this could offer both the faster startup of HTTP/2+, combined with the lower processing overhead for the db of a TCP stream. Comparing the graphs of 10 queries, it _appears_ that processing HTTP imposes a ~15 ms overhead over raw bytes. (Theoretically, the CONNECT proxy has to obey the host and port in the :authority pseudoheader, but in your case, the proxy would just ignore it, and hook up the edge function to whatever compute node it pleased.)
We experimented with SELECT 1, so most of the latency is transit
I'm assuming the most likely difference is in Pg processing an HTTP request over processing raw bytes, but there are other explanations, too. (Older/slower code pathway for HTTP in proxy, different intermediate node behavior for HTTP vs TCP from proxy to db, etc. Much less likely, but not impossible.)
The major advantage you'd get with HTTP/3 and QUIC is fewer round trips to start a conn, and no head-of-line blocking like with TCP (which might be irrelevant if the Pg protocol doesn't use simultaneous streams, dunno).
The disadvantage is you'd have to support QUIC in your proxy (and gmac said your server lib doesn't support it yet.)
Don't get hung up on HTTP/3 as the latest and greatest, HTTP/2 is probably still an improvement for you.
> Even when queries are sent from unrelated V8 isolates.
Heh, I've definitely written a few AWS Lambdas that exploited this sort of behavior. "Sure, this resource might not exist. But I'm going to check, just in case..."
A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
(HTTP/3 is ~90% the same as HTTP/2, just over a different transport layer, and ditching HTTP/2 prioritization mechanisms which you probably won't care about.)
> A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
Based on the network diagram in your article, it looks like the edge functions speak HTTP to the proxy, upgrade that to WS, and then the proxy routes raw TCP to the conn pool/db.
If so, CONNECT would be the same. The edge function sends an HTTP CONNECT request, the proxy would set it up, and afterwards, both hops are using the same transports as with websockets. The edge function would send HTTP/2 DATA frames wrapping the raw bytes, the proxy would unwrap those bytes, and then route the bytes over TCP to the conn pool/db.
If I understand your setup correctly, HTTP/2 CONNECT would be applicable everywhere WS is.
Neon is slow and flaky I am thankful you are making changes but strongmanning your technical choice when you are wrong is concerning as a user. I still use Neon due to not being able to move back to heroku. I also you keep making open source more.
The edge is in general not the users browser.
So you can do a TLS/postgres connection instead of adding additional layers.
Sure you want to probably run a patched pg client/server which has various optimizations to reduce the number of round trips when opening a connection, but that also applies to any HTTP/WebSocket wrapper.
Sure, sometimes you can. But a key use-case for a serverless database is connecting from a serverless compute environment, and not all serverless compute environments support TCP connections.
That's why Neon offers this serverless driver, which supports WebSockets and now also http.
[1]: https://github.com/neondatabase/serverless [2]: https://github.com/MatteoGioioso/serverless-pg