> Amazon Aurora DSQL is a serverless distributed SQL database with virtually unlimited scale, the highest availability, and zero infrastructure management.
I will ignore their claimed almost unlimited scalability (which I somewhat believe as it's in their best interest to make it expensive fast), and "highest" (as compared to what?) availability. But my pet peeve is zero infrastructure management. This is blatantly false and it always was - when they first announced their cloud (at that time called "web") services, when they marketed their solutions as "serverless", and now. You need to have well-trained staff to maintain your AWS infrastructure otherwise you will come across many problems.
I think this is way, way too pessimistic and unfair. When they say zero, it's like a food item that says zero calories. It might have up to five calories while still saying zero, but that's very different than three hundred.
Managing a SQL Database at scale is more like four thousand calories daily, if I continue stretching this analogy. :)
I don't disagree about the service itself having relatively few calories (I'd argue it's far more then 5, but that's a minor point). The reality is that you need a ton of other things to have it running properly - from a properly created and maintained AWS account, to proper IAM roles with correct permissions, to correct deployment/modification setup and other resources as a database rarely exists in a vacuum. And because Amazon decided not to implement spending cap, if you don't have the right competences, you are at risk losing money - and this is especially true for serverless services. I still remember the folks who set the same bucket for lambda notifications look with panic as their bill increases dramatically as they watch it growing, not knowing how to stop the whole process.[0]
> I will ignore their claimed almost unlimited scalability (which I somewhat believe as it's in their best interest to make it expensive fast)
I would believe it can scale your cost easily, but I'd be surprised if it can scale well in terms of capability. How many million QPS is "virtually unlimited scale"? The 10k row transaction limit and no FKs (or even views???) also just make this thing sound completely non-serious.
In my experience, multi-region databases are used for HA/DR more so than for "writing locally from multiple places". What this means is that the multi-region "consensus cost" is a far easier problem. It looks like DSQL is aiming to change that paradigm: the question is, will this actually mean multiple-region writes become democratized finally? or will this remain more of a database industry marketing talking point that essentially no one uses?
If that’s what you want, try cockroachdb serverless. Scales from zero in well under 1s even for multiregion. Also actually has quite good Postgres support these days.
It's frustrating that there's no pricing information. The tech looks cool and all, but without knowing how much it's going to cost there's no way to really evaluate it.
It's really surprising that they don't have a page with information ready to go live when the announcement happens. E.g. the Nova AI models they announced look neat, but the Bedrock page doesn't mention them at all and the page for them has links to non-existent documentation.
Stay Tuned
We're sorry. The documentation you are looking for is not yet available. Please check back soon.
-The Amazon Web Services Documentation Team
The only announcement I want from AWS about databases is that the price for RDS is going down. Not that they have new chip that is more expensive but offers better price/performance, but that my bill is actually going to drop.
I don't trust them enough to use non-portable technology like this until they give me confidence they are committed to lowering prices.
> The only announcement I want from AWS about databases is that the price for RDS is going down.
If the price hasn't been going up in line with inflation then it's been going down in real terms. Not as nice as the number going down, too, but it's not nothing. Over time AWS prices have significantly declined in real terms simply by not having been raised in nominal value.
Of all the services AWS has, RDS is one of the best values and lowest risks. The peace of mind and reliability it gives you on something as important as a DB is well worth the cost.
> Sooo they're finally launching a Spanner contender?
It may be even better in one important way: (if I understand correctly) for Spanner, you have to provision servers -- and keep them running. So GCP version of Spanner really makes no sense for anything that isn't huge.
This sounds more like "Spanner as a Service" -- which is something I've always wanted (for my hobby projects, no less :)
I don't know what I want exactly but I'm thinking along the lines that SQL is already doing a lot so it would make the most sense to start with a database interface and augment from there, to try to build a system to handle all the common forms of durable storage used by applications.
The type of situation I'm thinking about is for example storing a blob in S3, storing metadata and a reference to the blob's path in a database row, sending a message into a queue to trigger some async processing, and updating a cache. It would be nice to be able to do this through a single API or service, and it would be really nice to do all this within some type of transaction abstraction that would allow all operations to pass or fail collectively, really really nice if the whole thing could be pay-as-you and scale horizontally-ish on shared infrastructure without managing nodes or slots or whatever.
I'm not a Postgres user so I don't know how far you can get currently and I should probably look into it in detail. Coordinating blob/ject storage, database, and pubsub operations is a pain point for me presently. I think that overall system design is going to prevent a database-type system from being a good idea for blob storage but I would still like to see someone try to put three systems in a trenchcoat and try to make it work behind one interface.
I was just trying to get a sense for where the line of demarcation was in your mind. PG has "foreign data wrappers"[1] that allow one to treat external ... whatever ... as if it were a table or procedure within PG. Just stupid powerful, IMHO. It is FDW-specific whether "transaction" means anything to the foreign system, so that may break your mental model but could still get you very close (e.g. BEGIN; UPDATE s3 SET ...; ROLLBACK; may not do anything sensible)
Identifies as PG 16.5
No views/triggers/sequences
No foreign key constraints
No extensions
No NOTIFY ("ERROR: Function pg_notify not supported")
No nested transactions
No json(b)
It feels very disingenuous to say "Postgres compatible" and have this as a missing feature set. I'm sure they'd quickly argue it's wire compatibility, but even then it's a slippery slope and wire compatible is left open to however the person wants to interpret it.
There is no 'standard' or 'spec' for what makes something Postgres wire compatible.
This feels like a strong overreach on the marketing front to leverage the love people have for Postgres to help boost what they've built. That is not to say there isn't hard and quality engineering in here, but slapping Postgres compatible on it feels lazy at best.
> I'm sure they'd quickly argue it's wire compatibility, but even then it's a slippery slope and wire compatible is left open to however the person wants to interpret it.
I actually think that they'd argue they intend to close the feature gap for full Postgres semantics over time. Indeed their marketing was a bit wishful, but on Bluesky, Marc Brooker (one of the developers on the project) said they reused the parser, planner, and optimizer from Postgres: https://bsky.app/profile/marcbrooker.bsky.social/post/3lcghj...
That means they actually have a very good shot at approaching reasonably full Postgres compatibility (at a SQL semantics level, not just at the wire protocol level) over time.
Agreed, but it seems likely that people will casually pick this up instead of actual Postgres without fully appreciating the potential for consistency issues. I'd feel happier if AWS marketing applied your description of the value proposition - 'DynamoSQL'.
The quotas page is always the most interesting for any aws service.
10Mb transaction size and 5min transaction duration limits are interesting limitations that kill any "everything lives in the db" notions that one might have.
A fully transactional, scalable, distributed data store + queue + db + data stream with triggers is my endgame
Also, the 128mb limit on transaction temporary storage seems _very_ limiting for big ol' analytic queries
You say this as though the kind of people interested in a serverless distributed SQL product know what FK Constraints are, let alone have interest in using them.
This but unironically - if I’m in the market for a geo distributed hyper scale DB, I genuinely and sincerely have no interest in foreign key constraints. They are pretty much useless, if not actively harmful, at scale.
Disagree heavily that they’re useless at scale. Both PlanetScale and Citus support them (though the former doesn’t support it on sharded DBs yet), as well as TiDB (alpha), and Yugabyte.
As far as performance hits go, I guarantee I could find a half dozen other, larger problems that would have a bigger impact than disabling FK constraints.
It is of course possible to maintain referential integrity without them, but it requires much more diligence from devs, and I have yet to see it implemented well anywhere.
Until a single DB instance is serving well in excess of 100K QPS, I wouldn’t be concerned with the impact of FK constraints. Let RDBMS do what it’s good at – keeping your data safe.
I didnt necessarily mean scale as req/sec, although that is a factor, more system complexity & num of engineers. In a large enough companies tech ecosystem you tend to evolve a few characteristics:
- many data stores. even if you're not doing microservices persay, you're almost certainly going to end up with different domains/buis units/systems having their own data stores. FK constraints obviously dont work across system boundaries. They could be vendor systems too, not even just your own services.
- soft deletes. you rarely hard delete root objects, so you dont need the FK constraint to protect those dangling refs
- eventual consistentency/idempotency - you end up creating data in different systems as part of a workflow (eg signup), and you have to tolerate partial creation for resiliency.
- some sort of ORM or sql framework that makes it very difficult to create a leaf row with an invalid FK parent pointer.
So in a fundamentally distributed large scale tech ecosystem, FK constraints end up protecting against.... not much. Ive worked at three large scale (10m+ users) companies which all ended up deleting all their FK constraints. And... nothing happened, nothing ever went wrong due to their absence. Its theoretical protection, not practically needed IMHO.
> FK constraints obviously dont work across system boundaries.
Not to their full extent, but they can still be used. At the simplest level, it is of course entirely possible to give different services their own schema in a given database, and FK constraints are supported across schemata in both MySQL and Postgres. Vertical scaling can take you enormously far with properly architected schemata and queries.
A more flexible, but still easy to reason about way to accomplish this is to have local versions of certain tables in each DB. This can be manually implemented (though this is not an easy problem to solve, for a variety of reasons), or by using something like Citus [0], which accomplishes this using 2PC. This is of course slower, but if your data model is carefully designed, it can be managed.
> soft deletes.
Sure, but now you have a new problem - needing to add a `is_deleted` or `deleted_at` column to a bunch of tables, and indexing that column on every table. In Postgres you might get away with this by using `DEFAULT [FALSE, NULL]` (respectively), and then creating a partial index with `...WHERE <column> IS NOT [FALSE, NULL]`; that way the index size stays reasonable, and the cardinality isn't as horrible (well, it is for bools, but since you're only using it as a filter, it can work OK). Also, of course, you have to include this predicate in most queries.
> Its theoretical protection, not practically needed IMHO.
Different subjective experiences, of course, but IME it very much saves you time and headaches.
So it’s basically a CQL/Cassandra alternative. It’s interesting there’s also no comparison page to Cassandra or Amazon Keyspaces.
I also now don’t get why they are emphasizing PostgreSQL here when they should be emphasizing CQL compatibility. Being compatible with Postgres wire protocol but not being relational is strange.
What? Cassandra is a very different technology semantically. From data design to consistency models, to transactions, it’s entirely different, don’t know how you drew this conclusion.
My guess is from the customer's perspective, DSQL seems to have too many limitations, making it feel more like an enhanced version of a NoSQL database with SQL semantics, ACID transactions, and multi-region capabilities rather than a truly distributed version of a relational database. It seems the DSQL team doesn't fully understand why relational databases remain widely popular today. What makes them great is their flexibility, which lets them handle all kinds of use cases and adapt as things change. But all the limits on transaction size, column size, etc.., and too many missing capabilities pretty much take away all the big advantages relational databases usually offer.
Postgres compatibility depends a bit on what you rely on.
This doesn't appear to be postgres under the hood, more wire-compatible.
>Aurora DSQL is PostgreSQL compatible, which means that it provides identical behavior for most supported features, identical query results for all SQL features, and supports many popular PostgreSQL drivers and tools with minor configuration changes. Supported SQL expressions return identical data in query results, including sort order, scale and precision for numeric operations, and equivalence for string operations. With a few documented exceptions, such as synchronous replication, no-lock concurrency control, and asynchronous DDL execution, Aurora DSQL behaves comparably to PostgreSQL.
Aurora DSQL supports core relational features like ACID transactions, secondary indexes, joins, insert, and updates. See Supported SQL expressions for an overview of supported SQL features.
Aurora DSQL doesn't support all PostgreSQL features. For more information, see Unsupported PostgreSQL features.
> Except for some basic wire compatibility with the postgres protocol, I'd hardly call this a "database", and more a key-value store.
Hopefully that keeps the pricing reasonable. :)
But seriously, for a smaller CRUD app, this could be sufficient, even "magical," if the price is right. For my part though, the lack of multiple databases per cluster puts multi-tenant systems completely off the table. Now that you mention it, I almost wonder if this is a giant hack on top of Valkey/Redis...
I don't think that will matter. This feels much more like DynamoDB where you're charged for GB used/stored and no infra cost, so no reason to nest databases
You do not need logical database isolation if you have even stronger isolation with virtual clusters. The pricing and quota will be important factor for that use case.
I don't know, I manage a PHP app with about 10K users. If it doesn't cost much more, just paying to have the peace of mind that I never need to think about it again as my app grows, would not be a bad trade. This is assuming, of course, that the price is not extortionate.
I work with a lot of financial services applications, and they are most definitely in the "simple CRUD" with regards to the database realm, but with extremely high throughput and multiregional.
Having it be Postgres compatible is the big selling point.
Despite this number of limitations, I imagine that it's still far more usable from an application than DynamoDB for many use-cases. Adding Jsonb support would make it even more competitive for some.
Yep, people not seeing the usecase have not tried to build a serverless product on AWS and found themselves stuck between cheap, performant but denormalized ddb or whatever the fuck aurora serverless v2 is supposed to be.
I do not want to denormalize my data model, i do not have a high performance usecase, I simply want infrastructure that scales with usage (down to 0) and a flexible normalized model I can build more on top of easily.
DDB is great and all until you're asked to add 3 filters and orderby feature and suddenly you're adding elastic search to your project
Seems like it's mostly AWS' answer to Cloudflare D2, but using Postgres as the frame of reference instead of sqlite. (I'm not really sure what to call Postgres in this situation since it's so limited it's clearly not any full version of Postgres in any respect. Postgres-ish)
AWS tends to prioritize performance and scalability over functionality, which is reflected in the design of DynamoDB, SimpleDB, and now DSQL. I'm also not a big fan of this style. It doesn't give customers the flexibility to choose their own trade-offs like Spanner does and assumes that customers can't make these kinds of decisions on their own.
The fact that it uses "Firecracker microVMs" and a few other things makes me think this is an S3-backed Lambda-based database that emulates Postgres on the wire.
Especially when you consider the recent feature additions to S3.
I won't even consider this without pricing, I can't imagine how anyone could.
Serverless Aurora is stupid-expensive so I can't imagine this is going to be cheap but I keep hunting for that perfect DB that can scale to 0 (or very low) and be priced well.
Right now neon.tech is what I'm using and I'm very happy but this looks like it would be interesting IF the pricing is good.
I'm also on Neon. Seems like the primary benefit of this would be multi-region support, which Neon doesn't have. But I do wonder if multi-AZ is actually just enough, has there ever really been a time when all AZs have gone down?
we have multi-region disaster recovery and replicas coming in 2025. Switch over time will be greater than this active-active system but the overall latency for Neon should be much less on a consistent basis.
Oof, press release proof readers must have been hit by layoffs
> With its innovative active-active distributed architecture, Aurora DSQL is designed for 99.99% availability in single-Region configuration and 99.999% in multi-Region configuration, with an innovative active-active, distributed architecture, making it ideal for building highly-available applications
We get that its architecture is innovative and active-active
Can Amazon please get RDS product management ducks in a row? You got RDS For PostgreSQL, Aurora PostgreSQL (provisioned), Aurora Serverless v1, Aurora Serverless v2, Aurora PostgreSQL Limitless Database, Aurora Global Database, and now Aurora DSQL (with PostgreSQL compatibility). It's very confusing, the docs are confusing, and the even the tools are confused.
I think this is pretty big contrast to something like S3 where the product feels far more clearly and coherently managed, and where they have released lots of big improvements/changes over the years without disrupting or muddying the core product in major ways.
Except you can't, because it will just misundertand you or say something hopelessly generic.
Even asking it something on the same page that it's advertised 'now supports asking about' completely failed for me - think it was ENIs or something; just said something completely generic, not at all about the instances I'd asked about which it advertised being able to do (and I happened to have a concurrent need for).
As a very naive person myself, isn’t “being very confusing” one of the main points of AWS? They know they are big and nobody will get fired for using AWS. So, the more confusing their products are, the more benefits for them (e.g., because you don’t know how much you will pay, because you are misusing their products, because perhaps you don’t actually need their products, etc)
So, any manager at AWS probably thinks this is all working very fine.
AWS is confusing because they have a lot of tools that differ in dimensions that 80% of the users don't understand.
If you don't understand, for example, the difference between MySQL and PostgresQL then RDS/PostgresQL and RDS/mysql are the same. How does aurora mysql and aurora PostgresQL differ from standard RDS versions? Well, you have to actually read and understand...or watch some yoyo's YouTube video explainer.
AWS is complicated because it offers a lot. Do you want a satellite downlink? AWS has that. How about a private 5G cell network? Yep. How about a distributed data store with basically fixed read latency? Well, there are a bunch of different options depending on what you want.
Because you're obviously not used to being in the enterprise space, you think that confusion is what they want. They don't. In the space where people spend the big bucks being totally clear is what every good vendor strives for. You're in a multi-year relationship with these companies, and you want them to succeed. AWS isn't Oracle.
Obviously there are tons of shops that will sell people the wrong thing. But then another shop comes along and takes their business (which is one project I'm involved with now - they have like some ridiculous RDS bill that shouldn't be more than $500/mo).
Is it? Every solution has some advantages, switching between them is relatively easy (especially if you can tolerate some writing downtime), and IMO it's not that confusing if you have at least some experience in the space.
Otherwise if you're just staring from scratch, I'd pick provisioned aurora if your load is steady and serverless v2 if not. Then IO optimized if the db doesn't fit into memory and non-IO-optimized until it does.
It's not just selecting the service. It's also trying to decipher which parts what docs apply to when reading. Or how different services interact. Or getting any understanding what "Aurora" means in different contexts.
Just random example I was poking around AWS Backup recently, and there you have option to enable Backup for different services. Among the other options, you have separate toggles for "Aurora" and "RDS".
Overall the whole RDS API has lots of weirdness because there are all these different things (and combinations of things). Another random example, configuring Data HTTP API, you got this thing:
> This operation applies only to Aurora Serverless v2 and provisioned DB clusters. To enable the HTTP endpoint for Aurora Serverless v1 DB clusters, use the EnableHttpEndpoint parameter of the ModifyDBCluster operation.
At least that specific case was nicely documented now. Not all the pitfalls are thst clear.
* RDS is a managed relational database service. You got a database? We'll run it in the cloud for you. Exact same bits and bytes as you're running locally.
* Aurora is Amazon's own relational database. You can't run it yourself, only with Amazon. It can pretend to be either Postgres or MySQL. And it'll be cheaper and faster and have higher availability. But it won't be the exact same bits and bytes as your own Postgres, so there's some risks.
So far the stuff we said runs on clusters. You pick how big and how powerful and how many and in which AZ and configure how to scale it.
This has fairly pragmatic limits before at a certain point sharding and continuous horizontal scaling just gets too hard.
* Until Aurora Limitless that is which is, well probably not limitless (I don't know) but effectively.
* But you're still configuring cluster sizes and scaling policies. If you don't want to do that, Aurora has a Serverless option. It's the same Aurora but now you don't have to worry about scaling it yourself. The first version was Aurora Serverless but people said it wasn't very good.
* So they put out a 2nd one which is great and scales to zero.
* Now if you have globally distributed customers or care a lot about resiliency, you probably want your database in multiple regions. And you're going to be setting up eventual consistent updates for that. To make that easier there is Aurora Global Database, which is the same Aurora, but now with cross region replication.
* But it's not strongly consistent across regions. Aurora DSQL is. It's an even more bespoke version of Postgres. Actually it's not a relational database one at all, it just pretends to be one. But it uses atomic clocks and shit to cheat the cap theorem and give you global distribution and resiliency with strong consistency.
So in conclusion there's basically only 3 things here:
* RDS which is an unopinionated way of running a relational database in the Cloud
* Aurora, Amazon's highly opinionated relational database. That has Global, Serverless, and Limitless configuration options.
* And Aurora DSQL which is not Aurora a relational database at all but plays one on TV. But it gets to have the best of all worlds - SQL and NoSQL.
They probably should have called it something new but Aurora has good brand recognition. People know and trust Aurora.
In the long term all will probably continue to exist depending on where you are in your cloud journey. But I also expect that in the fullness of time if you're building a new Cloud native application and you don't have to worry about legacy migrations, you'll probably choose either DynamoDB or Aurora DSQL in 99.9% of cases.
>Aurora is Amazon's own relational database. You can't run it yourself, only with Amazon. It can pretend to be either Postgres or MySQL. And it'll be cheaper and faster and have higher availability. But it won't be the exact same bits and bytes as your own Postgres, so there's some risks.
That doesn't ring true to me. From my understanding rds aurora mostly replaces the storage engines of these DBs and the frontent mostly follows upstream releases and is compatible with them.
You're spot on. GP is repeating the common misconception that "Aurora" is a distinct, from-the-ground-up relational database system which can "pretend" to be either MySQL or Postgres by reimplementing their protocols/front-end from scratch. But that isn't the case at all.
In reality, Aurora MySQL is clearly heavily based on the MySQL codebase, but with major changes specifically to InnoDB's persistence and MVCC layers. Aurora MySQL implements a physical replication system using shared storage, undo, and redo logging. The rest is otherwise compatible with MySQL because it is using a lot of MySQL code.
I haven't read up on Aurora Postgres, but I'd bet it's something similar being applied to the Postgres codebase.
It's entirely possible the custom storage components may share some common code between Aurora MySQL and Aurora Postgres, but a custom storage layer alone isn't a "relational database" which "pretends" to be other DBs. (In contrast, it appears that DSQL actually might be that – a custom system with a reimplementation of Postgres wire protocol.)
Not trying to knock Aurora btw, it's really interesting technology and is quite an achievement by Amazon. But the constant confusion about what Aurora even is creates some support burdens in the DB ecosystem.
> f you're building a new Cloud native application and you don't have to worry about legacy migrations, you'll probably choose either DynamoDB or Aurora DSQL in 99.9% of cases.
If cloud-native means spending your life in AWS, that is.
But I use "cloud native" in a way that actually makes sense, not the way that GCP dubbed Kubernetes and all of it's ecosystem of friends "cloud-native".
"Native" development on a desktop OS means using primitives specific to that OS. As opposed to using something like Java Applets or Electron which will give you cross-platform compatibility, but you're not developing "natively" for the platform. That has the obvious pros and cons to each way.
Kubernetes is very much the Electron way: Rely on a generic high-level abstraction that will then create awkward bindings of running on abstract generic servers - whether they're in your data center or in the cloud.
Calling that "cloud-native" is some Orwellian post-truthness. I hate it.
If you want to develop in the cloud and make the most of it, use Azure, use GCP, use AWS, I don't care (not true: AWS is the best; use AWS). Just use the cloud provider's NATIVE primitives.
Otherwise, you might as well go back to running your own data centers, what are you even doing.
(Note: i'm just some guy. You don't have to listen to me. It's just what I think)
Very nice write up! It makes even more sense if you compare with Azure and see that they have the same kind of offering with the exact same breakdown, i.e. Azure Database for PostgreSQL/MySQL which is the managed relational database solution, Azure SQL databases which is the in-house one (i.e. SQL Server modified for Cloud) and promise cheaper/faster/higher HA because that's their own opinionated technology, with serverless and hyperscale (i.e. limitless) configuration options. They don't have DSQL equivalent yet, but their NoSQL offer (Cosmos DB) already have SQL query language support and with some modifications & improvements could be their "best of all worlds" equivalent.
It appears to be a recent generational trend to openly flaunt one's ignorance...Admit you've never read the documentation but prefer to "learn on the job," criticizing Certifications, and having no patience for attending formal training.
145 comments
[ 3.1 ms ] story [ 224 ms ] threadI will ignore their claimed almost unlimited scalability (which I somewhat believe as it's in their best interest to make it expensive fast), and "highest" (as compared to what?) availability. But my pet peeve is zero infrastructure management. This is blatantly false and it always was - when they first announced their cloud (at that time called "web") services, when they marketed their solutions as "serverless", and now. You need to have well-trained staff to maintain your AWS infrastructure otherwise you will come across many problems.
Managing a SQL Database at scale is more like four thousand calories daily, if I continue stretching this analogy. :)
[0] They disabled the trap a few weeks ago, finally: https://aws.amazon.com/about-aws/whats-new/2024/10/aws-lambd...
I would believe it can scale your cost easily, but I'd be surprised if it can scale well in terms of capability. How many million QPS is "virtually unlimited scale"? The 10k row transaction limit and no FKs (or even views???) also just make this thing sound completely non-serious.
https://aws.amazon.com/rds/aurora/serverless/
https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
This appears wire-compatible with postgres but does things differently like locking.
Depends a lot on how you use postgres as to whether its drop-in compatible or not.
AWS' recent "serverless" products, Aurora Serverless v2 and MSK Serverless for example, had been very misleading with "serverless" title.
I hope it is not a lie this time.
However, the latency scaling from zero is still painful, "up to 15 seconds".
EDIT: Your link is up now too.
I don't trust them enough to use non-portable technology like this until they give me confidence they are committed to lowering prices.
If the price hasn't been going up in line with inflation then it's been going down in real terms. Not as nice as the number going down, too, but it's not nothing. Over time AWS prices have significantly declined in real terms simply by not having been raised in nominal value.
https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
I'm itching to read more details into what this actually is under the marketing blab.
Seeing the No to configurable for some of the settings is the most telling hard limits that we can see up front.
Some Very Noteable ones are: Maximum size of all data modified within a write transaction: 10 MiB Maximum transaction time: 5 minutes
https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
It may be even better in one important way: (if I understand correctly) for Spanner, you have to provision servers -- and keep them running. So GCP version of Spanner really makes no sense for anything that isn't huge.
This sounds more like "Spanner as a Service" -- which is something I've always wanted (for my hobby projects, no less :)
Unless I misundertood, anyway.
It provides strong consistency for cross-region transactions
The type of situation I'm thinking about is for example storing a blob in S3, storing metadata and a reference to the blob's path in a database row, sending a message into a queue to trigger some async processing, and updating a cache. It would be nice to be able to do this through a single API or service, and it would be really nice to do all this within some type of transaction abstraction that would allow all operations to pass or fail collectively, really really nice if the whole thing could be pay-as-you and scale horizontally-ish on shared infrastructure without managing nodes or slots or whatever.
I'm not a Postgres user so I don't know how far you can get currently and I should probably look into it in detail. Coordinating blob/ject storage, database, and pubsub operations is a pain point for me presently. I think that overall system design is going to prevent a database-type system from being a good idea for blob storage but I would still like to see someone try to put three systems in a trenchcoat and try to make it work behind one interface.
https://github.com/turbot/steampipe#steampipe-plugins and https://steampipe.io/docs/steampipe_postgres/overview may be relevant, although watch out for Steampipe's license
https://github.com/topics/foreign-data-wrapper and https://github.com/topics/fdw are some other examples
1: https://www.postgresql.org/docs/17/fdwhandler.html (although strictly speaking that page is for _authoring_ FDW, not a tl;dr of the concept)
There is no 'standard' or 'spec' for what makes something Postgres wire compatible.
This feels like a strong overreach on the marketing front to leverage the love people have for Postgres to help boost what they've built. That is not to say there isn't hard and quality engineering in here, but slapping Postgres compatible on it feels lazy at best.
I actually think that they'd argue they intend to close the feature gap for full Postgres semantics over time. Indeed their marketing was a bit wishful, but on Bluesky, Marc Brooker (one of the developers on the project) said they reused the parser, planner, and optimizer from Postgres: https://bsky.app/profile/marcbrooker.bsky.social/post/3lcghj...
That means they actually have a very good shot at approaching reasonably full Postgres compatibility (at a SQL semantics level, not just at the wire protocol level) over time.
They'd save space by listing the supported features. It's not useless, but no existing application would run on it.
> Unsupported objects
> * Databases - Aurora DSQL supports only one database per cluster at this time.
> * Views
> * Temporary Tables
> * Triggers
> * Types
> * Tablespaces
> * UDFs / Functions other than functions using language = SQL
> Unsupported constraints
> Foreign keys
> Exclusion constraints
> Limitations
> ...
> A transaction cannot modify more than 10,000 rows
If you want a distributed PostgreSQL-lookalike, look at CockroachDB.
So it lacks Serializable isolation also, but I guess that's the secret to how it "offers the fastest distributed SQL reads and writes".
The quotas page is always the most interesting for any aws service.
10Mb transaction size and 5min transaction duration limits are interesting limitations that kill any "everything lives in the db" notions that one might have.
A fully transactional, scalable, distributed data store + queue + db + data stream with triggers is my endgame
Also, the 128mb limit on transaction temporary storage seems _very_ limiting for big ol' analytic queries
Wat?
As far as performance hits go, I guarantee I could find a half dozen other, larger problems that would have a bigger impact than disabling FK constraints.
It is of course possible to maintain referential integrity without them, but it requires much more diligence from devs, and I have yet to see it implemented well anywhere.
Until a single DB instance is serving well in excess of 100K QPS, I wouldn’t be concerned with the impact of FK constraints. Let RDBMS do what it’s good at – keeping your data safe.
- many data stores. even if you're not doing microservices persay, you're almost certainly going to end up with different domains/buis units/systems having their own data stores. FK constraints obviously dont work across system boundaries. They could be vendor systems too, not even just your own services.
- soft deletes. you rarely hard delete root objects, so you dont need the FK constraint to protect those dangling refs
- eventual consistentency/idempotency - you end up creating data in different systems as part of a workflow (eg signup), and you have to tolerate partial creation for resiliency.
- some sort of ORM or sql framework that makes it very difficult to create a leaf row with an invalid FK parent pointer.
So in a fundamentally distributed large scale tech ecosystem, FK constraints end up protecting against.... not much. Ive worked at three large scale (10m+ users) companies which all ended up deleting all their FK constraints. And... nothing happened, nothing ever went wrong due to their absence. Its theoretical protection, not practically needed IMHO.
Not to their full extent, but they can still be used. At the simplest level, it is of course entirely possible to give different services their own schema in a given database, and FK constraints are supported across schemata in both MySQL and Postgres. Vertical scaling can take you enormously far with properly architected schemata and queries.
A more flexible, but still easy to reason about way to accomplish this is to have local versions of certain tables in each DB. This can be manually implemented (though this is not an easy problem to solve, for a variety of reasons), or by using something like Citus [0], which accomplishes this using 2PC. This is of course slower, but if your data model is carefully designed, it can be managed.
> soft deletes.
Sure, but now you have a new problem - needing to add a `is_deleted` or `deleted_at` column to a bunch of tables, and indexing that column on every table. In Postgres you might get away with this by using `DEFAULT [FALSE, NULL]` (respectively), and then creating a partial index with `...WHERE <column> IS NOT [FALSE, NULL]`; that way the index size stays reasonable, and the cardinality isn't as horrible (well, it is for bools, but since you're only using it as a filter, it can work OK). Also, of course, you have to include this predicate in most queries.
> Its theoretical protection, not practically needed IMHO.
Different subjective experiences, of course, but IME it very much saves you time and headaches.
[0]: https://www.citusdata.com
I also now don’t get why they are emphasizing PostgreSQL here when they should be emphasizing CQL compatibility. Being compatible with Postgres wire protocol but not being relational is strange.
>Aurora DSQL is PostgreSQL compatible, which means that it provides identical behavior for most supported features, identical query results for all SQL features, and supports many popular PostgreSQL drivers and tools with minor configuration changes. Supported SQL expressions return identical data in query results, including sort order, scale and precision for numeric operations, and equivalence for string operations. With a few documented exceptions, such as synchronous replication, no-lock concurrency control, and asynchronous DDL execution, Aurora DSQL behaves comparably to PostgreSQL.
Aurora DSQL supports core relational features like ACID transactions, secondary indexes, joins, insert, and updates. See Supported SQL expressions for an overview of supported SQL features.
Aurora DSQL doesn't support all PostgreSQL features. For more information, see Unsupported PostgreSQL features.
https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
No pricing that I can find
No temporary tables, no foreign keys, no views, no more than 10k rows in a transaction.
Except for some basic wire compatibility with the postgres protocol, I'd hardly call this a "database", and more a key-value store.
https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
Hopefully that keeps the pricing reasonable. :)
But seriously, for a smaller CRUD app, this could be sufficient, even "magical," if the price is right. For my part though, the lack of multiple databases per cluster puts multi-tenant systems completely off the table. Now that you mention it, I almost wonder if this is a giant hack on top of Valkey/Redis...
I don't think that will matter. This feels much more like DynamoDB where you're charged for GB used/stored and no infra cost, so no reason to nest databases
That makes it very easy to manage access to sensitive data.
Having it be Postgres compatible is the big selling point.
Who else would use it? Not sure, gaming perhaps?
I do not want to denormalize my data model, i do not have a high performance usecase, I simply want infrastructure that scales with usage (down to 0) and a flexible normalized model I can build more on top of easily.
DDB is great and all until you're asked to add 3 filters and orderby feature and suddenly you're adding elastic search to your project
Not that it matters but it's D1 but I noticed people sometimes call it D2, maybe by analogy with R2.
This is the first release of something new and groundbreaking.
Especially when you consider the recent feature additions to S3.
Serverless Aurora is stupid-expensive so I can't imagine this is going to be cheap but I keep hunting for that perfect DB that can scale to 0 (or very low) and be priced well.
Right now neon.tech is what I'm using and I'm very happy but this looks like it would be interesting IF the pricing is good.
we have multi-region disaster recovery and replicas coming in 2025. Switch over time will be greater than this active-active system but the overall latency for Neon should be much less on a consistent basis.
> With its innovative active-active distributed architecture, Aurora DSQL is designed for 99.99% availability in single-Region configuration and 99.999% in multi-Region configuration, with an innovative active-active, distributed architecture, making it ideal for building highly-available applications
We get that its architecture is innovative and active-active
I think this is pretty big contrast to something like S3 where the product feels far more clearly and coherently managed, and where they have released lots of big improvements/changes over the years without disrupting or muddying the core product in major ways.
Not to worry -- you can use Rufus...uh "Q"....uhh...Nova...to summarize the docs into a decision tree on which confusing choice to use!
Even asking it something on the same page that it's advertised 'now supports asking about' completely failed for me - think it was ENIs or something; just said something completely generic, not at all about the instances I'd asked about which it advertised being able to do (and I happened to have a concurrent need for).
As a very naive person myself, isn’t “being very confusing” one of the main points of AWS? They know they are big and nobody will get fired for using AWS. So, the more confusing their products are, the more benefits for them (e.g., because you don’t know how much you will pay, because you are misusing their products, because perhaps you don’t actually need their products, etc)
So, any manager at AWS probably thinks this is all working very fine.
If you don't understand, for example, the difference between MySQL and PostgresQL then RDS/PostgresQL and RDS/mysql are the same. How does aurora mysql and aurora PostgresQL differ from standard RDS versions? Well, you have to actually read and understand...or watch some yoyo's YouTube video explainer.
AWS is complicated because it offers a lot. Do you want a satellite downlink? AWS has that. How about a private 5G cell network? Yep. How about a distributed data store with basically fixed read latency? Well, there are a bunch of different options depending on what you want.
Because you're obviously not used to being in the enterprise space, you think that confusion is what they want. They don't. In the space where people spend the big bucks being totally clear is what every good vendor strives for. You're in a multi-year relationship with these companies, and you want them to succeed. AWS isn't Oracle.
Obviously there are tons of shops that will sell people the wrong thing. But then another shop comes along and takes their business (which is one project I'm involved with now - they have like some ridiculous RDS bill that shouldn't be more than $500/mo).
Otherwise if you're just staring from scratch, I'd pick provisioned aurora if your load is steady and serverless v2 if not. Then IO optimized if the db doesn't fit into memory and non-IO-optimized until it does.
Just random example I was poking around AWS Backup recently, and there you have option to enable Backup for different services. Among the other options, you have separate toggles for "Aurora" and "RDS".
Overall the whole RDS API has lots of weirdness because there are all these different things (and combinations of things). Another random example, configuring Data HTTP API, you got this thing:
https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/AP...
> This operation applies only to Aurora Serverless v2 and provisioned DB clusters. To enable the HTTP endpoint for Aurora Serverless v1 DB clusters, use the EnableHttpEndpoint parameter of the ModifyDBCluster operation.
At least that specific case was nicely documented now. Not all the pitfalls are thst clear.
This is deprecated.
* Aurora is Amazon's own relational database. You can't run it yourself, only with Amazon. It can pretend to be either Postgres or MySQL. And it'll be cheaper and faster and have higher availability. But it won't be the exact same bits and bytes as your own Postgres, so there's some risks.
So far the stuff we said runs on clusters. You pick how big and how powerful and how many and in which AZ and configure how to scale it.
This has fairly pragmatic limits before at a certain point sharding and continuous horizontal scaling just gets too hard.
* Until Aurora Limitless that is which is, well probably not limitless (I don't know) but effectively.
* But you're still configuring cluster sizes and scaling policies. If you don't want to do that, Aurora has a Serverless option. It's the same Aurora but now you don't have to worry about scaling it yourself. The first version was Aurora Serverless but people said it wasn't very good.
* So they put out a 2nd one which is great and scales to zero.
* Now if you have globally distributed customers or care a lot about resiliency, you probably want your database in multiple regions. And you're going to be setting up eventual consistent updates for that. To make that easier there is Aurora Global Database, which is the same Aurora, but now with cross region replication.
* But it's not strongly consistent across regions. Aurora DSQL is. It's an even more bespoke version of Postgres. Actually it's not a relational database one at all, it just pretends to be one. But it uses atomic clocks and shit to cheat the cap theorem and give you global distribution and resiliency with strong consistency.
So in conclusion there's basically only 3 things here:
* RDS which is an unopinionated way of running a relational database in the Cloud
* Aurora, Amazon's highly opinionated relational database. That has Global, Serverless, and Limitless configuration options.
* And Aurora DSQL which is not Aurora a relational database at all but plays one on TV. But it gets to have the best of all worlds - SQL and NoSQL.
They probably should have called it something new but Aurora has good brand recognition. People know and trust Aurora.
In the long term all will probably continue to exist depending on where you are in your cloud journey. But I also expect that in the fullness of time if you're building a new Cloud native application and you don't have to worry about legacy migrations, you'll probably choose either DynamoDB or Aurora DSQL in 99.9% of cases.
It is public information?
See at this timestamp: https://youtu.be/LY7m5LQliAo?t=3507
That doesn't ring true to me. From my understanding rds aurora mostly replaces the storage engines of these DBs and the frontent mostly follows upstream releases and is compatible with them.
In reality, Aurora MySQL is clearly heavily based on the MySQL codebase, but with major changes specifically to InnoDB's persistence and MVCC layers. Aurora MySQL implements a physical replication system using shared storage, undo, and redo logging. The rest is otherwise compatible with MySQL because it is using a lot of MySQL code.
I haven't read up on Aurora Postgres, but I'd bet it's something similar being applied to the Postgres codebase.
It's entirely possible the custom storage components may share some common code between Aurora MySQL and Aurora Postgres, but a custom storage layer alone isn't a "relational database" which "pretends" to be other DBs. (In contrast, it appears that DSQL actually might be that – a custom system with a reimplementation of Postgres wire protocol.)
Not trying to knock Aurora btw, it's really interesting technology and is quite an achievement by Amazon. But the constant confusion about what Aurora even is creates some support burdens in the DB ecosystem.
If cloud-native means spending your life in AWS, that is.
But I use "cloud native" in a way that actually makes sense, not the way that GCP dubbed Kubernetes and all of it's ecosystem of friends "cloud-native".
"Native" development on a desktop OS means using primitives specific to that OS. As opposed to using something like Java Applets or Electron which will give you cross-platform compatibility, but you're not developing "natively" for the platform. That has the obvious pros and cons to each way.
Kubernetes is very much the Electron way: Rely on a generic high-level abstraction that will then create awkward bindings of running on abstract generic servers - whether they're in your data center or in the cloud.
Calling that "cloud-native" is some Orwellian post-truthness. I hate it.
If you want to develop in the cloud and make the most of it, use Azure, use GCP, use AWS, I don't care (not true: AWS is the best; use AWS). Just use the cloud provider's NATIVE primitives.
Otherwise, you might as well go back to running your own data centers, what are you even doing.
(Note: i'm just some guy. You don't have to listen to me. It's just what I think)
It appears to be a recent generational trend to openly flaunt one's ignorance...Admit you've never read the documentation but prefer to "learn on the job," criticizing Certifications, and having no patience for attending formal training.
https://brooker.co.za/blog/2024/12/03/aurora-dsql
Specifically, transaction latency is constant relative to transaction statement count, even across region, so I guess that's something
Maximum storage GB per cluster: 100GB Maximum size of all data modified within a write transaction 10 MiB Max: 10K rows per transaction
I imagine that number is tunable up to multi digit tb if you your monthly bill is worth Jassy's attention
https://docs.aws.amazon.com/aurora-dsql/latest/userguide/CHA...
The more interesting (i.e. constraining) limits are transaction size (both row count and data size) and transaction time, which aren't configurable.