Cool to see this (I’d somehow missed the early access link until last night).
Your storage strategy makes sense - building from first principles. To be honest, I figured you’d not done that though, and I imagine this makes portability a bit harder.
I’m eager to play with the service, but I wonder how local access patterns (e.g. dev workflows) differ - upon the right abstraction it shouldn’t really matter, right? I.e. a conn is a conn and a cursor is a cursor - the storage is opaque to the client. But does this somehow fall over with transactions and connection limits?
Please email beta@neon.tech and we will bump you up on the waitlist. Portability is key - we can't fork Postgres.
Apart from branching it's the same Postgres - so you can develop locally against vanilla Postgres. We are thinking how to connect local and cloud environments. What would be some good patterns here?
Splitting the architecture across compute/storage and then pageserver/walserver seems like a great design decision because the pieces fit snugly onto the underlying layers of cloud services. The design is general enough that it could work for other block-device-based databases (~all of them) so you could put a SQLite or MySQL compute layer on top of the storage layer instead of postgres. Discussion: https://github.com/neondatabase/neon/discussions/1958
The fact that they're targeting vanilla postgres is also intriuging because that means it will be a lot easier to integrate with other interesting PgSQL-based technologies; EdgeDB in particular seems like it would be a pretty great fit.
7 comments
[ 2.9 ms ] story [ 33.3 ms ] threadYour storage strategy makes sense - building from first principles. To be honest, I figured you’d not done that though, and I imagine this makes portability a bit harder.
I’m eager to play with the service, but I wonder how local access patterns (e.g. dev workflows) differ - upon the right abstraction it shouldn’t really matter, right? I.e. a conn is a conn and a cursor is a cursor - the storage is opaque to the client. But does this somehow fall over with transactions and connection limits?
Apart from branching it's the same Postgres - so you can develop locally against vanilla Postgres. We are thinking how to connect local and cloud environments. What would be some good patterns here?
The fact that they're targeting vanilla postgres is also intriuging because that means it will be a lot easier to integrate with other interesting PgSQL-based technologies; EdgeDB in particular seems like it would be a pretty great fit.
I was discussing with Wes that we can put arrow compute on top of neon storage.
This sounds cool, but what happens if the server crashes? How do you prevent losing the buffered WAL? Do you have WAL of WAL or something?